-- Logs begin at Wed 2022-06-15 09:21:14 UTC, end at Wed 2022-06-15 11:14:48 UTC. -- Jun 15 10:00:59.647715 dv-cinder-845806-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 15 10:00:59.656443 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 15 10:00:59.658347 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 15 10:00:59 2022] *** Jun 15 10:00:59.658347 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 15 10:00:59.658347 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 15 10:00:59.658347 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: nodename: dv-cinder-845806-2-iscsi Jun 15 10:00:59.658347 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: machine: x86_64 Jun 15 10:00:59.658347 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: clock source: unix Jun 15 10:00:59.658347 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: pcre jit disabled Jun 15 10:00:59.658347 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: detected number of CPU cores: 4 Jun 15 10:00:59.658347 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: current working directory: / Jun 15 10:00:59.658347 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: detected binary path: /usr/bin/uwsgi-core Jun 15 10:00:59.658347 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: your processes number limit is 47680 Jun 15 10:00:59.659066 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: your memory page size is 4096 bytes Jun 15 10:00:59.659066 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: detected max file descriptor number: 2048 Jun 15 10:00:59.659066 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: lock engine: pthread robust mutexes Jun 15 10:00:59.659066 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: thunder lock: enabled Jun 15 10:00:59.659066 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 15 10:00:59.659066 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 15 10:00:59.702781 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: Python main interpreter initialized at 0x562cd73ae9c0 Jun 15 10:00:59.702781 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: python threads support enabled Jun 15 10:00:59.702781 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: your server socket listen backlog is limited to 100 connections Jun 15 10:00:59.702781 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: your mercy for graceful operations on workers is 90 seconds Jun 15 10:00:59.703163 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: mapped 403077 bytes (393 KB) for 2 cores Jun 15 10:00:59.703238 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: *** Operational MODE: preforking *** Jun 15 10:00:59.703473 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: *** uWSGI is running in multiple interpreter mode *** Jun 15 10:00:59.703473 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: spawned uWSGI master process (pid: 80489) Jun 15 10:00:59.703680 dv-cinder-845806-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 15 10:00:59.708985 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: spawned uWSGI worker 1 (pid: 80502, cores: 1) Jun 15 10:00:59.708985 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: spawned uWSGI worker 2 (pid: 80503, cores: 1) Jun 15 10:00:59.708985 dv-cinder-845806-2-iscsi devstack@keystone.service[80489]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 15 10:01:05.246623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 10:01:05.246623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: from cryptography.utils import int_from_bytes Jun 15 10:01:05.247432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 10:01:05.247432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: from cryptography.utils import int_from_bytes Jun 15 10:01:05.252009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 10:01:05.252009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: from cryptography.utils import int_from_bytes Jun 15 10:01:05.252491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 15 10:01:05.252491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: from cryptography.utils import int_from_bytes Jun 15 10:01:05.296977 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 10:01:05.297181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 10:01:05.297377 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] command line args: [] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 10:01:05.297649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 10:01:05.297828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ================================================================================ {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 10:01:05.298083 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] admin_token = **** {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.298330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] config_dir = [] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.298518 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.298700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] config_source = [] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.298881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] debug = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.299082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.300332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.300332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.300480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 15 10:01:05.300554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.300619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 15 10:01:05.300619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] command line args: [] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 15 10:01:05.300727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.300789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 15 10:01:05.300850 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] list_limit = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.300917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ================================================================================ {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 15 10:01:05.301150 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] log_config_append = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.301150 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.301303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] admin_token = **** {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.301363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] log_dir = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.301673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] config_dir = [] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.301744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] log_file = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.301808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.301870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.301932 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] config_source = [] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.301988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.302048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] debug = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.302048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.302187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.302252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.302313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.302413 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.302537 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.302616 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.302689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.302871 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.302955 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.303108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.303201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] list_limit = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.303355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] log_config_append = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.303497 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.303627 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.303709 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.303826 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] log_dir = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.303944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.304025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] log_file = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.304138 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.304212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.304332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.304415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.304492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.304611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.304739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.304817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.304922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.305128 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.305258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.305375 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.305452 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] publish_errors = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.305622 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.305704 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.305858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.305982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.306063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.306132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.306283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.306365 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.306519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.306610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.306757 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] standard_threads = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.306833 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.306992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.307078 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.307237 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.307315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.307415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.307593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] use_journal = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.307750 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.307829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] use_json = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.307933 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] use_stderr = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.308010 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] publish_errors = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.308110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] use_syslog = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.308186 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.308343 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.308420 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.308546 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.308628 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.308794 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.308876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.309072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.309154 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.309314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] standard_threads = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.309398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.309548 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.309632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.309794 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.309878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.310025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.310099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.310167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] use_journal = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.310300 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] use_json = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.310376 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.310476 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] use_stderr = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.310621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.310703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] use_syslog = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.310779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.310872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 15 10:01:05.311020 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.311129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.311259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.311371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.311510 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.311632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.311746 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.311873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.311994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.312133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.312242 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.312372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.312481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.312624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.312737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.312863 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.313011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.313134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.313262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.313384 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.313499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.313632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.313754 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.313869 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.314015 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.314102 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.314270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.314354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.314519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.314603 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.314751 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.314836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.314983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.315060 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.315805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.316046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.316122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.316372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.316473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.316628 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.316731 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.316870 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.316948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.317160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.317246 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.317425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.317509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.317683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.317759 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.317923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.318001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.318155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.318236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.318390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.318466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.318623 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.318708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.318855 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.318932 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.319087 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.319164 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.319318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.319396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.319549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.319643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.319781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.319848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.320016 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.320084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.320263 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.320361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.320491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.320563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.320736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.320848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.320989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.321071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.321201 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.321277 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.321421 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.321504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.321651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.321726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.321883 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.321975 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.322116 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.322206 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.322347 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.322474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.322579 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.322656 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.322815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.322908 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.323047 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.323131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.323276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.323356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.323504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.323589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.323752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.323831 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.323989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.324075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.324222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.324313 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.324452 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.324530 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.324690 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.324777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.324926 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.325109 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.325232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.325312 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.325410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.325488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.325645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.325729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.325876 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.325954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.326127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.326206 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.326365 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.326454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.326603 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.326680 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.326833 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.326922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.327066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.327152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.327313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.327402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.327544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.327627 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.327776 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.327971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.328050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.328120 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.328244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.328321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.328481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.328567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.328718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.328796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.328951 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.329064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.329205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.329286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.329445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.329523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.329681 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.329764 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.329922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.329998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.330157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.330235 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.330395 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.330476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.330632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.330709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.330926 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.331530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.331782 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.331865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.332021 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.332100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.332260 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.332339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.332497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.332575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.332741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.332817 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.333007 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.333101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.333234 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.333323 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.333470 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.333714 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.333953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.334191 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.334428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.334661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.334893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.335032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.335118 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.335312 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.335390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.335571 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.335648 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.335812 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.335906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.336051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.336129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.336301 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.336379 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.336539 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.336629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] auth.external = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.336799 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.336885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.337126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.337370 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.337609 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] auth.password = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.337849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] auth.token = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.338095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.338337 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.338589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.338668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.338826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.338935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.339081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.339191 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] auth.external = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.339332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.339442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.339565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.339757 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.339842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] credential.caching = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.340006 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.340091 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.340248 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] auth.password = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.340327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.340490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] auth.token = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.340568 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.340739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.340819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.341010 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.341124 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.341251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.341340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.341493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.341573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.341751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.341830 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.342000 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.342086 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.342235 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.342314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.342478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] credential.caching = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.342566 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.342731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.342817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.342985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.343070 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.343239 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.343318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.343480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.343559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.343723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] federation.caching = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.343802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.343961 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.344040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.344210 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.344287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.344461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.344543 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.344695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.344774 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.344944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.345050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.345208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.345294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.345466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.345545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.345706 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.345793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.345969 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.346048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.346207 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.346288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.346973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.347160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] federation.caching = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.347241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity.caching = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.347415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.347685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.347765 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.347836 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.347915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.347988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.348132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.348211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.348385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.348472 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.348620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.348700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.348858 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.348938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.349126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.349208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.349380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.349460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.349615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.349694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.349848 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.349928 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.350081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.350161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity.caching = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.350328 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.350407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.350577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.350654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.350817 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.350905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.351056 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.351141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.351304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.351382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.351535 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.351615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.351776 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.351856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.352028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.352106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.352258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.352337 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.352496 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.352576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.352728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.352805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.353017 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.353100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.353228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.353308 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.353470 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.353549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.353718 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.353798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.353949 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.354026 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.354197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.354275 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.354444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.354525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.354682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.354761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.354924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.355010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.355160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.355239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.355389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.355466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.355620 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.355706 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.355850 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.355927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.356093 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.356173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.356331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.356418 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.356575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.356654 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.356804 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.356882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.357061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.357158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.357312 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.357391 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.357547 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.357624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.357780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.357869 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.358011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.358092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.358248 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.358325 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.358487 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.358575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.358722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.358801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.358956 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.359033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.359193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.359279 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.359422 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.359500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.359662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.359741 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.359897 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.359983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.360142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.360219 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.360387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.360464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.360619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.360706 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.360851 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.360930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.361116 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.361195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.361347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.361438 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.361585 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.361664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.361819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.361903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.362095 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.362678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.362858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.363018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.363090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.363314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.363392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.363548 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.363629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.363796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.363875 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.364046 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.364125 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.364294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.364373 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.364529 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.364606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.364757 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.364836 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.365015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.365099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.365254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.365333 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.365500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.365578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.365722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.365801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.365969 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.366046 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.366215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.366293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.366454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.366533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.366698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.366776 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.366931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.367008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.367168 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.367246 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.367404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.367482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.367642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.367721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.367878 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.367957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.368116 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.368195 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.368353 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.368432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.368579 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.368653 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.368810 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.368890 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.369075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] resource.caching = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.369155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.369331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.369412 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.369562 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.369639 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.369808 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.369885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.370043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.370122 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.370279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.370359 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.370520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.370606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.370766 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.370843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.371005 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.371083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.371235 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.371320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.371467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] role.caching = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.371545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.371719 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] resource.caching = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.371845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] role.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.371922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.372002 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.372169 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.372248 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.372414 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.372491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.372661 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.372753 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.372910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.373009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.373172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.373250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.373419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.373512 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.373655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.373733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.373897 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.373975 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.374128 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.374213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] role.caching = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.374375 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] role.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.374459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.374607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.374685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.374858 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.374944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.375106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.375184 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.375356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.375535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.375614 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.375775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.375854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.376028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.376112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.376284 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.376363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.376536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.376617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.376770 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.376851 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.377032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.377115 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.377282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.377362 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.377533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.377613 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.377856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.377957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.378183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.378263 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.378459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.378536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.378711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.378798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.378963 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.379042 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.379213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.379292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.379462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.379547 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.379696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.379775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.379931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.380015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.380164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.380251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.380400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] token.caching = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.380480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.380638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.380717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.380870 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.380977 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.381127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.381207 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.381381 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.381459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.381628 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.381718 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.381867 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.381946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.382105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.382183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.382344 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.382431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.382577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.382655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.382815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.382892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] token.caching = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.383050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.383136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.383285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.383362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.383524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.383602 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.383771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.383856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.384004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.384083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.384241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.384319 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.384486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.384573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.384730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.384848 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.connection = **** {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.385029 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.385131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.385238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.385317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.385478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.385559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.385717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.385796 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.385959 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.386037 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.386203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.386284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.386431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.386532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.386666 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.386744 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.386910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.386990 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.387146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.387225 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.connection = **** {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.387389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.387478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.387625 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.387705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.387875 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.387957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.388111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.388191 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.388345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.388422 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.388573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.388652 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.388805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.388883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.389061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.389141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.389225 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 10:01:05.389307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.389529 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.389775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.390013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.390263 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.390502 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.390747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.390980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.391223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.391458 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 15 10:01:05.391629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 15 10:01:05.507477 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.507652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.507849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.510636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.510873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.511104 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.516868 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.519793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.545716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.545952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.546536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.546771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.601283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.601654 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.650671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.650895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.650990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.651067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.717970 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.718183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.718269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.718403 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.718490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.718621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.718704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.718836 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.718920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.719048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.719130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.719264 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.742701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.742855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.742923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.742996 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.743146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.743256 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.743444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.743539 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.743613 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.743685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.743789 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.743885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.744002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.744085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.755250 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.755361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.767959 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.768142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.768233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.768856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.768939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.769112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.769309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.769393 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.769530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.769655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.769789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.769872 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.769945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.770158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 15 10:01:05.787605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.787812 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 10:01:05.788218 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.788421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 10:01:05.788872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.789161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.789425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.789685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.789768 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.789930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 10:01:05.790266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.790464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 15 10:01:05.790914 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.791173 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.791561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.791821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.800553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.800825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.801122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.801407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.803172 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.803449 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.803717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.803996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.810457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.812235 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.813886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.814169 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.814445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.814720 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.814996 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.815282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.815550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.815664 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.815819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.815975 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.816085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.816213 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.816354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.816488 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.816617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.816762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.817050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.817322 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.817589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.817853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.818121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.818382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.843658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.845314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.845593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.850483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.851489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.851788 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.852068 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.852150 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.852380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.857696 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.857975 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.858253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.859578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.859857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.864380 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.864667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.864948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.865250 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.865522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.865765 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.865848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.866046 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.866201 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.866314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.870604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.870889 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.871166 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.871447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.871834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.872109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.872375 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.872646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.884864 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.885159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.889666 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.891259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.891531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.893813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.894096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.896544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.899265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.900281 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.900570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.902851 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.905621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.906591 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.906889 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.907170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.907448 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.907721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.908003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.909792 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.913435 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.913729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.914011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.914310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.914567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.914842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.928073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.928357 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.928626 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.928888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.934165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.934447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.934713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.934972 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.936761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.938513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.938789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.942289 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.942571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.942811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.944306 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.944608 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.944690 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.944896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.945008 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.947953 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.948242 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.950009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.950309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.950589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.958905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.959186 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.959465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.959743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.960075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.960292 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.960565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.964570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.964846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.965161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.965441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.965715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.966250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.979049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.982372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.985100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.985668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.987390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.988432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.989379 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.989658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.989936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.991973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.993598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.995390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:05.995679 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.995958 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.997208 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:05.998829 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:06.002079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.002374 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.003190 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.003613 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.003818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.005464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:06.008796 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.009113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.009395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.009667 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.012906 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:06.013216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.013499 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.019568 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:06.019815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.020094 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.022114 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:06.022393 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.022672 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.022955 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.023224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.023513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.023784 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.024048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.024322 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.024692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.024974 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.025270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.025537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.025797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.027979 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 15 10:01:06.028262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.028535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.028803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.029095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.029380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.029649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.029920 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.030190 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.030553 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.030826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.031096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.031360 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.031623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 15 10:01:06.064623 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x562cd73ae9c0 pid: 80503 (default app) Jun 15 10:01:06.068708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec34271-8311-45d1-ad86-250cf89e6f2d None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:06.069020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec34271-8311-45d1-ad86-250cf89e6f2d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:06.069308 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec34271-8311-45d1-ad86-250cf89e6f2d None None] PATH_INFO: `/v3` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:06.070504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x562cd73ae9c0 pid: 80502 (default app) Jun 15 10:01:06.073527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1/1] 10.222.0.60 () {50 vars in 791 bytes} [Wed Jun 15 10:01:06 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:14.887674 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a90b5d-713b-4367-9d98-34aa61158aa0 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:14.888203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a90b5d-713b-4367-9d98-34aa61158aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:14.888203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a90b5d-713b-4367-9d98-34aa61158aa0 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:14.892477 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1/2] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:01:14.898088 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8087866-8ce5-46a6-bc7b-8b0e8a79736c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:14.898510 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8087866-8ce5-46a6-bc7b-8b0e8a79736c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:14.898646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8087866-8ce5-46a6-bc7b-8b0e8a79736c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:14.968686 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: INFO dbcounter [None req-a8087866-8ce5-46a6-bc7b-8b0e8a79736c None None] Registered counter for database keystone Jun 15 10:01:14.990830 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG oslo_db.sqlalchemy.engines [None req-a8087866-8ce5-46a6-bc7b-8b0e8a79736c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80503) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 10:01:14.997206 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writer thread running {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 10:01:15.182195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG passlib.handlers.bcrypt [None req-a8087866-8ce5-46a6-bc7b-8b0e8a79736c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80503) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 10:01:15.182570 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG passlib.handlers.bcrypt [None req-a8087866-8ce5-46a6-bc7b-8b0e8a79736c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80503) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 10:01:15.239871 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-a8087866-8ce5-46a6-bc7b-8b0e8a79736c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:15.241540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-a8087866-8ce5-46a6-bc7b-8b0e8a79736c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:15.369344 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 2/3] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 15 10:01:15.532990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a90b5d-713b-4367-9d98-34aa61158aa0 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:15.606322 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: INFO dbcounter [None req-34a90b5d-713b-4367-9d98-34aa61158aa0 None None] Registered counter for database keystone Jun 15 10:01:15.631841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG oslo_db.sqlalchemy.engines [None req-34a90b5d-713b-4367-9d98-34aa61158aa0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80502) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 15 10:01:15.638314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writer thread running {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 15 10:01:15.762135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a90b5d-713b-4367-9d98-34aa61158aa0 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:15.766471 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47060cb1-d99c-4d33-82b3-eaef03e5fb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:15.767150 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47060cb1-d99c-4d33-82b3-eaef03e5fb34 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:15.767438 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47060cb1-d99c-4d33-82b3-eaef03e5fb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:15.767720 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47060cb1-d99c-4d33-82b3-eaef03e5fb34 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:15.776273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47060cb1-d99c-4d33-82b3-eaef03e5fb34 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:15.828932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: WARNING keystone.common.rbac_enforcer.enforcer [None req-47060cb1-d99c-4d33-82b3-eaef03e5fb34 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 10:01:15.829219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47060cb1-d99c-4d33-82b3-eaef03e5fb34 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:15.840034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 2/4] 10.222.0.60 () {58 vars in 1324 bytes} [Wed Jun 15 10:01:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:01:15.846018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8087866-8ce5-46a6-bc7b-8b0e8a79736c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:15.865366 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8087866-8ce5-46a6-bc7b-8b0e8a79736c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:15.869845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0e83d-7476-43b9-8666-5c6369b4fb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:15.870962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0e83d-7476-43b9-8666-5c6369b4fb54 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:15.871246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0e83d-7476-43b9-8666-5c6369b4fb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:15.871530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0e83d-7476-43b9-8666-5c6369b4fb54 admin admin] PATH_INFO: `/v3/projects/8232a31392e948a78496aa9a0a9a81a0` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:15.873045 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0e83d-7476-43b9-8666-5c6369b4fb54 admin admin] RBAC: Authorizing `identity:get_project(project_id=8232a31392e948a78496aa9a0a9a81a0)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:15.925890 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: WARNING keystone.common.rbac_enforcer.enforcer [None req-85c0e83d-7476-43b9-8666-5c6369b4fb54 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 15 10:01:15.926763 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0e83d-7476-43b9-8666-5c6369b4fb54 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:15.931211 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 3/5] 10.222.0.60 () {56 vars in 1207 bytes} [Wed Jun 15 10:01:15 2022] GET /identity/v3/projects/8232a31392e948a78496aa9a0a9a81a0 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:18.061910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64025e-eae6-46e9-b5d7-2a30ff2b3cc8 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:18.062467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64025e-eae6-46e9-b5d7-2a30ff2b3cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:18.062467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64025e-eae6-46e9-b5d7-2a30ff2b3cc8 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:18.067200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 3/6] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:01:18.072859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d28633-b9e1-4b03-bb32-2eaf8789198d None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:18.073295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d28633-b9e1-4b03-bb32-2eaf8789198d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:18.073463 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d28633-b9e1-4b03-bb32-2eaf8789198d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:18.092491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-e1d28633-b9e1-4b03-bb32-2eaf8789198d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:18.094062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e1d28633-b9e1-4b03-bb32-2eaf8789198d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:18.102219 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 4/7] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 15 10:01:18.124840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64025e-eae6-46e9-b5d7-2a30ff2b3cc8 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:18.137654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64025e-eae6-46e9-b5d7-2a30ff2b3cc8 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:18.141929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a98fc-f159-4e68-81a5-8b00eea8e9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:18.142565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a98fc-f159-4e68-81a5-8b00eea8e9ed admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:18.143469 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a98fc-f159-4e68-81a5-8b00eea8e9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:18.143818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a98fc-f159-4e68-81a5-8b00eea8e9ed admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:18.151978 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a98fc-f159-4e68-81a5-8b00eea8e9ed admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:18.153824 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a98fc-f159-4e68-81a5-8b00eea8e9ed admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:18.163177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 4/8] 10.222.0.60 () {58 vars in 1324 bytes} [Wed Jun 15 10:01:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:01:18.168915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d28633-b9e1-4b03-bb32-2eaf8789198d None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:18.178475 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d28633-b9e1-4b03-bb32-2eaf8789198d None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:18.182880 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67148cd4-4306-4bfa-90ff-6d89ae58a6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:18.183751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67148cd4-4306-4bfa-90ff-6d89ae58a6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:18.184034 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67148cd4-4306-4bfa-90ff-6d89ae58a6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:18.184313 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67148cd4-4306-4bfa-90ff-6d89ae58a6b4 admin admin] PATH_INFO: `/v3/users/ede33ec052df4b3dab11128e172fd980` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:18.185743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67148cd4-4306-4bfa-90ff-6d89ae58a6b4 admin admin] RBAC: Authorizing `identity:get_user(user_id=ede33ec052df4b3dab11128e172fd980)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:18.187576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67148cd4-4306-4bfa-90ff-6d89ae58a6b4 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:18.189195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 5/9] 10.222.0.60 () {56 vars in 1198 bytes} [Wed Jun 15 10:01:18 2022] GET /identity/v3/users/ede33ec052df4b3dab11128e172fd980 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:20.396895 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d9e26-497c-4195-b808-6c3636f8b8b6 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:20.397432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d9e26-497c-4195-b808-6c3636f8b8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:20.397538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d9e26-497c-4195-b808-6c3636f8b8b6 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:20.398971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 5/10] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:01:20.404594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3c3a5-1695-4ee7-bd00-c97954bc022c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:20.405133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3c3a5-1695-4ee7-bd00-c97954bc022c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:20.405427 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3c3a5-1695-4ee7-bd00-c97954bc022c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:20.423301 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-10a3c3a5-1695-4ee7-bd00-c97954bc022c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:20.425288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-10a3c3a5-1695-4ee7-bd00-c97954bc022c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:20.426591 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872d5d0-8376-4cdd-8041-957a4a961a09 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:20.426891 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872d5d0-8376-4cdd-8041-957a4a961a09 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:20.427172 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9872d5d0-8376-4cdd-8041-957a4a961a09 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:20.428666 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 6/11] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:01:20.434325 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b35fa-30da-44a7-bd78-6a74bd18cc4b None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:20.434621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b35fa-30da-44a7-bd78-6a74bd18cc4b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:20.434907 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b35fa-30da-44a7-bd78-6a74bd18cc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:20.476180 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG passlib.handlers.bcrypt [None req-a65b35fa-30da-44a7-bd78-6a74bd18cc4b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80502) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 15 10:01:20.476611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG passlib.handlers.bcrypt [None req-a65b35fa-30da-44a7-bd78-6a74bd18cc4b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80502) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 15 10:01:20.508881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 6/12] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:20.522349 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a65b35fa-30da-44a7-bd78-6a74bd18cc4b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:20.524034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a65b35fa-30da-44a7-bd78-6a74bd18cc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:20.532214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3c3a5-1695-4ee7-bd00-c97954bc022c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:20.558701 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:20.627045 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 7/13] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:01:20.639424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3c3a5-1695-4ee7-bd00-c97954bc022c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:20.650348 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b35fa-30da-44a7-bd78-6a74bd18cc4b None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:20.675506 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f03d6-6e1f-4165-84ce-95c0f5012b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:20.676314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f03d6-6e1f-4165-84ce-95c0f5012b55 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:20.676846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f03d6-6e1f-4165-84ce-95c0f5012b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:20.677147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f03d6-6e1f-4165-84ce-95c0f5012b55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:20.688591 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f03d6-6e1f-4165-84ce-95c0f5012b55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-b80f03d6-6e1f-4165-84ce-95c0f5012b55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:20.700431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:20.701602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:20.702779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:20.702779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:20.702779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:20.702779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:20.702779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:20.702779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:20.702779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:20.702779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:20.702779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:20.702779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:20.702779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:20.702779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:01:20.702779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 7/14] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:01:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:01:20.708007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f03d6-6e1f-4165-84ce-95c0f5012b55 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:20.757468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b35fa-30da-44a7-bd78-6a74bd18cc4b None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:20.759834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:20.792987 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:20.797956 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14555907-f4ca-46cf-bbfa-bc8cbbf13fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:20.798862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14555907-f4ca-46cf-bbfa-bc8cbbf13fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:20.799255 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14555907-f4ca-46cf-bbfa-bc8cbbf13fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:20.799535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14555907-f4ca-46cf-bbfa-bc8cbbf13fa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:20.811775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f03d6-6e1f-4165-84ce-95c0f5012b55 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:20.847271 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb237434-f2bc-4f4d-8d1d-d3823afb5ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:20.847947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb237434-f2bc-4f4d-8d1d-d3823afb5ecf None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:20.848232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb237434-f2bc-4f4d-8d1d-d3823afb5ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:20.848506 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb237434-f2bc-4f4d-8d1d-d3823afb5ecf None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:20.849297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb237434-f2bc-4f4d-8d1d-d3823afb5ecf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:20.849663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb237434-f2bc-4f4d-8d1d-d3823afb5ecf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:20.851061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb237434-f2bc-4f4d-8d1d-d3823afb5ecf None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:20.863076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 8/15] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:20.868489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb237434-f2bc-4f4d-8d1d-d3823afb5ecf None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:20.876118 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14555907-f4ca-46cf-bbfa-bc8cbbf13fa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:20.877918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14555907-f4ca-46cf-bbfa-bc8cbbf13fa1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:20.967735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb237434-f2bc-4f4d-8d1d-d3823afb5ecf None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:20.979038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 8/16] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:01:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:01:20.984606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14555907-f4ca-46cf-bbfa-bc8cbbf13fa1 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:20.987526 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:21.004907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01714c-039e-4978-9b09-515e0c537c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:21.005867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01714c-039e-4978-9b09-515e0c537c97 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:21.006206 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01714c-039e-4978-9b09-515e0c537c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:21.006487 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01714c-039e-4978-9b09-515e0c537c97 None admin] PATH_INFO: `/v3/users/ede33ec052df4b3dab11128e172fd980` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:21.007807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01714c-039e-4978-9b09-515e0c537c97 None admin] RBAC: Authorizing `identity:get_user(user_id=ede33ec052df4b3dab11128e172fd980)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:21.009396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01714c-039e-4978-9b09-515e0c537c97 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:21.011095 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 9/17] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:20 2022] GET /identity/v3/users/ede33ec052df4b3dab11128e172fd980 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:21.016599 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01714c-039e-4978-9b09-515e0c537c97 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:21.032418 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:21.086976 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14555907-f4ca-46cf-bbfa-bc8cbbf13fa1 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:21.120454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c6935-2252-4a4b-b197-481fb0128a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:21.121295 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c6935-2252-4a4b-b197-481fb0128a6e None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:21.121407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01714c-039e-4978-9b09-515e0c537c97 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:21.121550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c6935-2252-4a4b-b197-481fb0128a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:21.121825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c6935-2252-4a4b-b197-481fb0128a6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:21.142254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c6935-2252-4a4b-b197-481fb0128a6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:21.144754 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c6935-2252-4a4b-b197-481fb0128a6e None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:21.155938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945f2b7-884a-47be-97a4-d14155dc8846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:21.156751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945f2b7-884a-47be-97a4-d14155dc8846 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:21.157068 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945f2b7-884a-47be-97a4-d14155dc8846 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:21.157352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c945f2b7-884a-47be-97a4-d14155dc8846 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:21.158551 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945f2b7-884a-47be-97a4-d14155dc8846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:21.160025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c945f2b7-884a-47be-97a4-d14155dc8846 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:21.161677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 10/18] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:01:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:21.165724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-8d0c6935-2252-4a4b-b197-481fb0128a6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:01:21.165724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:21.165724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:21.165724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:21.165724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:21.165724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:21.165724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:21.165724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:21.165724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:21.165724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:21.165724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:21.165724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:21.165724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:01:21.165724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:01:21.165724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:21.166642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:21.167466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:01:21.167466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:01:21.167466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:01:21.167466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:01:21.167466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 9/19] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:21.167922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945f2b7-884a-47be-97a4-d14155dc8846 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:21.172420 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c6935-2252-4a4b-b197-481fb0128a6e None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:21.229585 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:21.249000 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:21.269340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c945f2b7-884a-47be-97a4-d14155dc8846 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:21.274569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c6935-2252-4a4b-b197-481fb0128a6e None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:21.304232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a367c39-7ad1-4da9-9e84-7f7f352bc593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:21.304891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a367c39-7ad1-4da9-9e84-7f7f352bc593 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:21.305212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a367c39-7ad1-4da9-9e84-7f7f352bc593 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:21.305492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a367c39-7ad1-4da9-9e84-7f7f352bc593 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:21.306341 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a367c39-7ad1-4da9-9e84-7f7f352bc593 None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.domain.id=default, user.id=ede33ec052df4b3dab11128e172fd980) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:21.306660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a367c39-7ad1-4da9-9e84-7f7f352bc593 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:21.308115 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a367c39-7ad1-4da9-9e84-7f7f352bc593 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:21.308982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68174670-5112-4c43-b799-d1f8408eb77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:21.309654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68174670-5112-4c43-b799-d1f8408eb77d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:21.309943 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68174670-5112-4c43-b799-d1f8408eb77d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:21.310223 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68174670-5112-4c43-b799-d1f8408eb77d None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:21.310959 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68174670-5112-4c43-b799-d1f8408eb77d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:21.311306 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68174670-5112-4c43-b799-d1f8408eb77d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:21.312677 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68174670-5112-4c43-b799-d1f8408eb77d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:21.315885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 11/20] 10.222.0.60 () {56 vars in 1322 bytes} [Wed Jun 15 10:01:21 2022] GET /identity/v3/role_assignments?user.id=ede33ec052df4b3dab11128e172fd980&scope.domain.id=default&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:21.333653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 10/21] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:23.584923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73aaf2-f151-436c-97fd-66cb9fc21eca None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:23.585434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73aaf2-f151-436c-97fd-66cb9fc21eca None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:23.585505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73aaf2-f151-436c-97fd-66cb9fc21eca None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:23.586876 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 12/22] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:23.592221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000f92f-6a9d-45f5-ac5e-bb10253bb754 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:23.593090 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000f92f-6a9d-45f5-ac5e-bb10253bb754 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:23.593581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000f92f-6a9d-45f5-ac5e-bb10253bb754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:23.622042 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-4000f92f-6a9d-45f5-ac5e-bb10253bb754 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:23.623607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4000f92f-6a9d-45f5-ac5e-bb10253bb754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:23.718311 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 11/23] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:23.753613 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73aaf2-f151-436c-97fd-66cb9fc21eca None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:23.884243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73aaf2-f151-436c-97fd-66cb9fc21eca None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:23.934703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0bd05b-0fd0-48a2-8757-0a169cb972f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:23.935508 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bd05b-0fd0-48a2-8757-0a169cb972f0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:23.935793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bd05b-0fd0-48a2-8757-0a169cb972f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:23.936073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0bd05b-0fd0-48a2-8757-0a169cb972f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:23.951597 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0bd05b-0fd0-48a2-8757-0a169cb972f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-7f0bd05b-0fd0-48a2-8757-0a169cb972f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:23.959554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:23.960400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:23.961185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:23.961185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:23.961185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:23.961185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:23.961185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:23.961185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:23.961185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:23.961185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:23.961185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:23.961185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:23.961185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:23.961185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:01:23.961185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 13/24] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:01:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:01:23.966789 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000f92f-6a9d-45f5-ac5e-bb10253bb754 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:24.057986 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:24.114496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe4f90-afe8-443c-96a4-4d33d127d5ad None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:24.115196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe4f90-afe8-443c-96a4-4d33d127d5ad None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:24.115683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe4f90-afe8-443c-96a4-4d33d127d5ad None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:24.121660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 14/25] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:24.127385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1058ed6-8aa4-4886-ac6c-923a7bffacef None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:24.137115 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1058ed6-8aa4-4886-ac6c-923a7bffacef None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:24.137115 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1058ed6-8aa4-4886-ac6c-923a7bffacef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:24.138765 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:24.152925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000f92f-6a9d-45f5-ac5e-bb10253bb754 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:24.154877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-a1058ed6-8aa4-4886-ac6c-923a7bffacef None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:24.156312 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-a1058ed6-8aa4-4886-ac6c-923a7bffacef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:24.222001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93d5b4-0875-4011-99ff-a57816ec1033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:24.222723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93d5b4-0875-4011-99ff-a57816ec1033 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:24.223191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93d5b4-0875-4011-99ff-a57816ec1033 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:24.223470 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93d5b4-0875-4011-99ff-a57816ec1033 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:24.224615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93d5b4-0875-4011-99ff-a57816ec1033 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:24.224934 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93d5b4-0875-4011-99ff-a57816ec1033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:24.226528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93d5b4-0875-4011-99ff-a57816ec1033 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:24.245658 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 12/26] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:24.252904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93d5b4-0875-4011-99ff-a57816ec1033 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:24.313818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 15/27] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:01:24.358536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1058ed6-8aa4-4886-ac6c-923a7bffacef None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:24.461726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93d5b4-0875-4011-99ff-a57816ec1033 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:24.483002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:24.516699 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cc36f-2d66-426b-97f0-4b9d6eddbe0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:24.518598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cc36f-2d66-426b-97f0-4b9d6eddbe0d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:24.518598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cc36f-2d66-426b-97f0-4b9d6eddbe0d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:24.518836 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cc36f-2d66-426b-97f0-4b9d6eddbe0d None admin] PATH_INFO: `/v3/users/ede33ec052df4b3dab11128e172fd980` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:24.520470 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cc36f-2d66-426b-97f0-4b9d6eddbe0d None admin] RBAC: Authorizing `identity:get_user(user_id=ede33ec052df4b3dab11128e172fd980)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:24.522019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cc36f-2d66-426b-97f0-4b9d6eddbe0d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:24.523757 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 13/28] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:24 2022] GET /identity/v3/users/ede33ec052df4b3dab11128e172fd980 => generated 247 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:24.525250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d2292-6d37-43c8-81f6-07d88592f42b None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:24.525687 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d2292-6d37-43c8-81f6-07d88592f42b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:24.526129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d2292-6d37-43c8-81f6-07d88592f42b None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:24.527613 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 14/29] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:24.528919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fa14e-5716-45f9-9cde-02f117c02b04 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:24.529381 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fa14e-5716-45f9-9cde-02f117c02b04 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:24.529786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fa14e-5716-45f9-9cde-02f117c02b04 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:24.532057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 15/30] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:24.533247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fa14e-5716-45f9-9cde-02f117c02b04 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:24.564243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1058ed6-8aa4-4886-ac6c-923a7bffacef None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:24.603958 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695be48e-7938-43ea-98ab-02b7451ccb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:24.605701 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695be48e-7938-43ea-98ab-02b7451ccb06 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:24.606202 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695be48e-7938-43ea-98ab-02b7451ccb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:24.606731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695be48e-7938-43ea-98ab-02b7451ccb06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:24.609064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:24.619167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695be48e-7938-43ea-98ab-02b7451ccb06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:24.621121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695be48e-7938-43ea-98ab-02b7451ccb06 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:24.641659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-695be48e-7938-43ea-98ab-02b7451ccb06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:01:24.641659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:24.641659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:24.641659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:24.641659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:24.641659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:24.641659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:24.641659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:24.641659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:24.641659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:24.641659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:24.641659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:24.641659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:01:24.641659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:01:24.641659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:24.642492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:24.643980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:01:24.643980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:01:24.643980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:01:24.643980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:01:24.646193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 16/31] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:24.647698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebb00b-3f3f-41ea-acd1-d0fd27fd3a16 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:24.648201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebb00b-3f3f-41ea-acd1-d0fd27fd3a16 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:24.648686 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebb00b-3f3f-41ea-acd1-d0fd27fd3a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:24.661713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fa14e-5716-45f9-9cde-02f117c02b04 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:24.669301 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-fdebb00b-3f3f-41ea-acd1-d0fd27fd3a16 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:24.670759 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-fdebb00b-3f3f-41ea-acd1-d0fd27fd3a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:24.701701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410addb5-b934-4365-9fee-a14680400be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:24.702508 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410addb5-b934-4365-9fee-a14680400be1 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:24.702803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410addb5-b934-4365-9fee-a14680400be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:24.703064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410addb5-b934-4365-9fee-a14680400be1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:24.704434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410addb5-b934-4365-9fee-a14680400be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:24.706259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410addb5-b934-4365-9fee-a14680400be1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:24.712397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 16/32] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:01:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:24.712397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398ae9a-21d9-4ed3-9a63-5bdd495c924e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:24.712397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398ae9a-21d9-4ed3-9a63-5bdd495c924e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:24.712397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398ae9a-21d9-4ed3-9a63-5bdd495c924e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:24.727945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-3398ae9a-21d9-4ed3-9a63-5bdd495c924e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:24.729278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3398ae9a-21d9-4ed3-9a63-5bdd495c924e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:24.753244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 17/33] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:24.754479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebb00b-3f3f-41ea-acd1-d0fd27fd3a16 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:24.773884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:24.807184 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 17/34] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:24.812976 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398ae9a-21d9-4ed3-9a63-5bdd495c924e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:24.891868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebb00b-3f3f-41ea-acd1-d0fd27fd3a16 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:24.915262 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:24.929267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398ae9a-21d9-4ed3-9a63-5bdd495c924e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:24.940514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992716d-2890-4aae-bc36-0af6cd974131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:24.941310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992716d-2890-4aae-bc36-0af6cd974131 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:24.941596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992716d-2890-4aae-bc36-0af6cd974131 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:24.941923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992716d-2890-4aae-bc36-0af6cd974131 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:24.943103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2992716d-2890-4aae-bc36-0af6cd974131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:24.943414 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2992716d-2890-4aae-bc36-0af6cd974131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:24.945144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2992716d-2890-4aae-bc36-0af6cd974131 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:24.969230 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 18/35] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:24.970525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259db055-0b6a-4aea-af86-400012427925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:24.972359 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259db055-0b6a-4aea-af86-400012427925 None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:24.972841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259db055-0b6a-4aea-af86-400012427925 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:24.973389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259db055-0b6a-4aea-af86-400012427925 None admin] PATH_INFO: `/v3/domains/default/users/ede33ec052df4b3dab11128e172fd980/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:24.974934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992716d-2890-4aae-bc36-0af6cd974131 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:24.975906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259db055-0b6a-4aea-af86-400012427925 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ede33ec052df4b3dab11128e172fd980, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:24.977705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259db055-0b6a-4aea-af86-400012427925 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:25.080657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992716d-2890-4aae-bc36-0af6cd974131 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:25.119017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465b738-4af1-4d7e-914d-21b8f374c1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:25.119703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465b738-4af1-4d7e-914d-21b8f374c1bd None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:25.119991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465b738-4af1-4d7e-914d-21b8f374c1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:25.120326 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465b738-4af1-4d7e-914d-21b8f374c1bd None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:25.121385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465b738-4af1-4d7e-914d-21b8f374c1bd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:25.122776 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0465b738-4af1-4d7e-914d-21b8f374c1bd None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:25.207963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 18/36] 10.222.0.60 () {58 vars in 1361 bytes} [Wed Jun 15 10:01:24 2022] PUT /identity/v3/domains/default/users/ede33ec052df4b3dab11128e172fd980/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:01:25.209590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259db055-0b6a-4aea-af86-400012427925 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:25.274179 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:25.355515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259db055-0b6a-4aea-af86-400012427925 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:25.407776 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a49f3d-d3bc-4401-8be6-9c2c17ea5e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:25.408905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a49f3d-d3bc-4401-8be6-9c2c17ea5e33 None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:25.409431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a49f3d-d3bc-4401-8be6-9c2c17ea5e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:25.409907 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a49f3d-d3bc-4401-8be6-9c2c17ea5e33 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:25.410915 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a49f3d-d3bc-4401-8be6-9c2c17ea5e33 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:25.413741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a49f3d-d3bc-4401-8be6-9c2c17ea5e33 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:25.503925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 19/37] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Jun 15 10:01:25 2022] POST /identity/v3/roles => generated 219 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:01:25.509513 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a49f3d-d3bc-4401-8be6-9c2c17ea5e33 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:25.593369 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 19/38] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Jun 15 10:01:24 2022] POST /identity/v3/roles => generated 225 bytes in 624 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:01:25.595130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f772ca-d081-405e-9aaf-2464ff155820 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:25.595426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f772ca-d081-405e-9aaf-2464ff155820 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:25.596308 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f772ca-d081-405e-9aaf-2464ff155820 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:25.602082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 20/39] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:25.610702 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c49bae-d5de-4e57-bcae-81b5fa46322d None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:25.611966 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c49bae-d5de-4e57-bcae-81b5fa46322d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:25.611966 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c49bae-d5de-4e57-bcae-81b5fa46322d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:25.650393 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-03c49bae-d5de-4e57-bcae-81b5fa46322d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:25.652028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-03c49bae-d5de-4e57-bcae-81b5fa46322d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:25.773830 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:25.886572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 21/40] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:25.892397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca545ad1-d4dd-44b0-8ca4-329b7113229e None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:25.894983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca545ad1-d4dd-44b0-8ca4-329b7113229e None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:25.894983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca545ad1-d4dd-44b0-8ca4-329b7113229e None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:25.895198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 22/41] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:25.906741 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c9ad-6956-4078-a6a8-740d139fc60b None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:25.907391 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c9ad-6956-4078-a6a8-740d139fc60b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:25.907831 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c9ad-6956-4078-a6a8-740d139fc60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:25.914003 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:25.922171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a49f3d-d3bc-4401-8be6-9c2c17ea5e33 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:25.933264 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-1db4c9ad-6956-4078-a6a8-740d139fc60b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:25.934734 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-1db4c9ad-6956-4078-a6a8-740d139fc60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:25.959934 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86da8ce-a756-4e7f-a745-8145670bad7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:25.960621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86da8ce-a756-4e7f-a745-8145670bad7a None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:25.960905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86da8ce-a756-4e7f-a745-8145670bad7a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:25.961221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86da8ce-a756-4e7f-a745-8145670bad7a None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:25.961952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86da8ce-a756-4e7f-a745-8145670bad7a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:25.963435 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86da8ce-a756-4e7f-a745-8145670bad7a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:26.009862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 23/42] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:26.010982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c9ad-6956-4078-a6a8-740d139fc60b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:26.092638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 20/43] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jun 15 10:01:25 2022] POST /identity/v3/projects => generated 301 bytes in 588 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:01:26.093463 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86da8ce-a756-4e7f-a745-8145670bad7a None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:26.122992 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:26.131067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c9ad-6956-4078-a6a8-740d139fc60b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:26.185178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f677d-7fbd-4972-a3be-603e02f083cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:26.186140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f677d-7fbd-4972-a3be-603e02f083cf None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:26.186504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f677d-7fbd-4972-a3be-603e02f083cf None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:26.186790 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f677d-7fbd-4972-a3be-603e02f083cf None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:26.187594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f677d-7fbd-4972-a3be-603e02f083cf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:26.189220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f677d-7fbd-4972-a3be-603e02f083cf None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:26.234535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86da8ce-a756-4e7f-a745-8145670bad7a None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:26.249000 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 24/44] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Jun 15 10:01:26 2022] POST /identity/v3/roles => generated 223 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:01:26.288941 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76abec33-7f5e-4a8c-a63b-2ffb24bf9fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:26.289798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abec33-7f5e-4a8c-a63b-2ffb24bf9fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:26.290088 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abec33-7f5e-4a8c-a63b-2ffb24bf9fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:26.290375 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76abec33-7f5e-4a8c-a63b-2ffb24bf9fb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:26.291621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76abec33-7f5e-4a8c-a63b-2ffb24bf9fb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:26.293198 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76abec33-7f5e-4a8c-a63b-2ffb24bf9fb8 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:26.295109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 21/45] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:01:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:26.302331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f677d-7fbd-4972-a3be-603e02f083cf None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:26.433456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f677d-7fbd-4972-a3be-603e02f083cf None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:26.469799 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d01e03-aef3-478a-95e9-a760da1128b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:26.470489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d01e03-aef3-478a-95e9-a760da1128b0 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:26.471475 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d01e03-aef3-478a-95e9-a760da1128b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:26.471769 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d01e03-aef3-478a-95e9-a760da1128b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:26.472495 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d01e03-aef3-478a-95e9-a760da1128b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:26.474010 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d01e03-aef3-478a-95e9-a760da1128b0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:26.555708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 25/46] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jun 15 10:01:26 2022] POST /identity/v3/projects => generated 312 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:01:27.735920 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2da8e-b622-4d62-bf4c-effd993d6cea None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:27.737001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2da8e-b622-4d62-bf4c-effd993d6cea None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:27.737001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2da8e-b622-4d62-bf4c-effd993d6cea None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:27.738359 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 22/47] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:27.744377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f191364-e47e-4ca5-a067-7d49485938ce None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:27.744824 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f191364-e47e-4ca5-a067-7d49485938ce None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:27.744948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f191364-e47e-4ca5-a067-7d49485938ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:27.762220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-0f191364-e47e-4ca5-a067-7d49485938ce None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:27.763362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0f191364-e47e-4ca5-a067-7d49485938ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:27.765425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:27.831603 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 26/48] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:27.854531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2da8e-b622-4d62-bf4c-effd993d6cea None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:27.871401 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:27.956971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2da8e-b622-4d62-bf4c-effd993d6cea None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:27.990719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947947d-9a06-4cc0-a8f9-c1bed6678d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:27.991563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947947d-9a06-4cc0-a8f9-c1bed6678d1c None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:27.991853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947947d-9a06-4cc0-a8f9-c1bed6678d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:27.992130 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947947d-9a06-4cc0-a8f9-c1bed6678d1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:28.003250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947947d-9a06-4cc0-a8f9-c1bed6678d1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-1947947d-9a06-4cc0-a8f9-c1bed6678d1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:28.012132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:28.012952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:28.013877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:28.013877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:28.013877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:28.013877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:28.013877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:28.013877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:28.013877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:28.013877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:28.013877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:28.013877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:28.013877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:28.013877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:01:28.013877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 23/49] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:01:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:28.019129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f191364-e47e-4ca5-a067-7d49485938ce None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:28.121809 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f191364-e47e-4ca5-a067-7d49485938ce None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:28.156110 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2698ee-a8f3-4d8e-b0a3-f45c46cf6a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:28.156785 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2698ee-a8f3-4d8e-b0a3-f45c46cf6a5e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:28.157113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2698ee-a8f3-4d8e-b0a3-f45c46cf6a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:28.157449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2698ee-a8f3-4d8e-b0a3-f45c46cf6a5e None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:28.158608 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2698ee-a8f3-4d8e-b0a3-f45c46cf6a5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:28.158608 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2698ee-a8f3-4d8e-b0a3-f45c46cf6a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:28.160058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2698ee-a8f3-4d8e-b0a3-f45c46cf6a5e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:28.165394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:28.169032 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 27/50] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:28.175599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947947d-9a06-4cc0-a8f9-c1bed6678d1c None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:28.253536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:28.278336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947947d-9a06-4cc0-a8f9-c1bed6678d1c None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:28.312649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be8ce7-f9a7-45f6-9101-8f142cfa82c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:28.313558 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be8ce7-f9a7-45f6-9101-8f142cfa82c7 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:28.313841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be8ce7-f9a7-45f6-9101-8f142cfa82c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:28.314116 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be8ce7-f9a7-45f6-9101-8f142cfa82c7 None admin] PATH_INFO: `/v3/users/ede33ec052df4b3dab11128e172fd980` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:28.316084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be8ce7-f9a7-45f6-9101-8f142cfa82c7 None admin] RBAC: Authorizing `identity:get_user(user_id=ede33ec052df4b3dab11128e172fd980)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:28.317710 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be8ce7-f9a7-45f6-9101-8f142cfa82c7 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:28.319355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 24/51] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:28 2022] GET /identity/v3/users/ede33ec052df4b3dab11128e172fd980 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:28.324715 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2698ee-a8f3-4d8e-b0a3-f45c46cf6a5e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:28.427066 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2698ee-a8f3-4d8e-b0a3-f45c46cf6a5e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:28.460849 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42112c4e-67e0-4634-8f20-b9fb2c0b66ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:28.461684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42112c4e-67e0-4634-8f20-b9fb2c0b66ab None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:28.461972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42112c4e-67e0-4634-8f20-b9fb2c0b66ab None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:28.462248 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42112c4e-67e0-4634-8f20-b9fb2c0b66ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:28.463431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42112c4e-67e0-4634-8f20-b9fb2c0b66ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:28.464893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42112c4e-67e0-4634-8f20-b9fb2c0b66ab None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:28.466545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 28/52] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:01:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:28.473314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be8ce7-f9a7-45f6-9101-8f142cfa82c7 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:28.579444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be8ce7-f9a7-45f6-9101-8f142cfa82c7 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:28.613784 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b94fa-c0dd-490e-913f-8507aac738ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:28.614434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b94fa-c0dd-490e-913f-8507aac738ed None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:28.614719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b94fa-c0dd-490e-913f-8507aac738ed None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:28.614995 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b94fa-c0dd-490e-913f-8507aac738ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:28.615835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b94fa-c0dd-490e-913f-8507aac738ed None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.domain.id=default, user.id=ede33ec052df4b3dab11128e172fd980) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:28.616149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b94fa-c0dd-490e-913f-8507aac738ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:28.617784 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b94fa-c0dd-490e-913f-8507aac738ed None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:28.627463 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 25/53] 10.222.0.60 () {56 vars in 1322 bytes} [Wed Jun 15 10:01:28 2022] GET /identity/v3/role_assignments?user.id=ede33ec052df4b3dab11128e172fd980&scope.domain.id=default&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:28.726600 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735606b-3c52-475a-a593-4e391e6043b7 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:28.727004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735606b-3c52-475a-a593-4e391e6043b7 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:28.727352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735606b-3c52-475a-a593-4e391e6043b7 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:28.728847 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 29/54] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:28.734527 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa83d7-491a-4743-a1cc-bc61ca35f767 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:28.734825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa83d7-491a-4743-a1cc-bc61ca35f767 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:28.735110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa83d7-491a-4743-a1cc-bc61ca35f767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:28.747479 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:28.754778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-22fa83d7-491a-4743-a1cc-bc61ca35f767 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:28.756469 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-22fa83d7-491a-4743-a1cc-bc61ca35f767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:28.830192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 26/55] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:28.853704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735606b-3c52-475a-a593-4e391e6043b7 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:28.921263 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:28.954694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735606b-3c52-475a-a593-4e391e6043b7 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:28.987660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec64d1-91f1-488c-9f15-4f9f834641d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:28.988383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec64d1-91f1-488c-9f15-4f9f834641d0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:28.988668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec64d1-91f1-488c-9f15-4f9f834641d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:28.988946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec64d1-91f1-488c-9f15-4f9f834641d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:28.990154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec64d1-91f1-488c-9f15-4f9f834641d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:28.991600 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec64d1-91f1-488c-9f15-4f9f834641d0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:28.993254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 30/56] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:01:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:28.998900 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa83d7-491a-4743-a1cc-bc61ca35f767 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:29.097870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa83d7-491a-4743-a1cc-bc61ca35f767 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:29.132355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dfd2a-1e62-4ed2-b49b-85ba423b4612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:29.133044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dfd2a-1e62-4ed2-b49b-85ba423b4612 None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:29.133330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dfd2a-1e62-4ed2-b49b-85ba423b4612 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:29.133607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dfd2a-1e62-4ed2-b49b-85ba423b4612 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:29.134317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dfd2a-1e62-4ed2-b49b-85ba423b4612 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:29.135779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dfd2a-1e62-4ed2-b49b-85ba423b4612 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:29.139798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:29.211573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 27/57] 10.222.0.60 () {60 vars in 1140 bytes} [Wed Jun 15 10:01:28 2022] POST /identity/v3/projects => generated 298 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:01:31.355302 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cff12-b2e5-4b9f-a7e9-68493042f06f None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:31.355827 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cff12-b2e5-4b9f-a7e9-68493042f06f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:31.355913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cff12-b2e5-4b9f-a7e9-68493042f06f None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:31.357306 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 31/58] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:31.363666 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a1555-ee30-4548-8194-6d763664f610 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:31.364107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a1555-ee30-4548-8194-6d763664f610 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:31.364239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a1555-ee30-4548-8194-6d763664f610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:31.381524 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b43a1555-ee30-4548-8194-6d763664f610 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:31.382709 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b43a1555-ee30-4548-8194-6d763664f610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:31.450234 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 28/59] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:01:31.474443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cff12-b2e5-4b9f-a7e9-68493042f06f None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:31.574939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cff12-b2e5-4b9f-a7e9-68493042f06f None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:31.607714 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00acd48-d9c9-4a21-9051-cd0f909a58c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:31.608436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00acd48-d9c9-4a21-9051-cd0f909a58c2 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:31.608723 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00acd48-d9c9-4a21-9051-cd0f909a58c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:31.609026 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00acd48-d9c9-4a21-9051-cd0f909a58c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:31.610189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00acd48-d9c9-4a21-9051-cd0f909a58c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:31.612314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00acd48-d9c9-4a21-9051-cd0f909a58c2 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:31.614035 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 32/60] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:01:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:31.619572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a1555-ee30-4548-8194-6d763664f610 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:31.720107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a1555-ee30-4548-8194-6d763664f610 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:31.738398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:31.754982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136cfcd-4661-479a-9689-8fb8b71f9aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:31.755659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136cfcd-4661-479a-9689-8fb8b71f9aac None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:31.755945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136cfcd-4661-479a-9689-8fb8b71f9aac None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:31.756224 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136cfcd-4661-479a-9689-8fb8b71f9aac None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:31.756948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136cfcd-4661-479a-9689-8fb8b71f9aac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:31.758513 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136cfcd-4661-479a-9689-8fb8b71f9aac None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:31.856086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 29/61] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:01:31 2022] POST /identity/v3/users => generated 275 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:01:35.244683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a1b7a-11cd-47ac-bffb-155f16299e10 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:35.245252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a1b7a-11cd-47ac-bffb-155f16299e10 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:35.245325 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a1b7a-11cd-47ac-bffb-155f16299e10 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:35.246886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 33/62] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:35.252740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28fd5b-5252-4677-8199-0e90b5ad7ede None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:35.253400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28fd5b-5252-4677-8199-0e90b5ad7ede None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:35.253400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28fd5b-5252-4677-8199-0e90b5ad7ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:35.276867 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-bd28fd5b-5252-4677-8199-0e90b5ad7ede None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:35.282547 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-bd28fd5b-5252-4677-8199-0e90b5ad7ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:35.340169 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ba5b8-de3f-4c72-b349-e1cb27ed8271 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:35.340789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ba5b8-de3f-4c72-b349-e1cb27ed8271 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:35.341297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ba5b8-de3f-4c72-b349-e1cb27ed8271 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:35.344150 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 34/63] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:35.349249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac9afb-96d1-465c-bd18-3311659ec275 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:35.349821 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac9afb-96d1-465c-bd18-3311659ec275 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:35.350297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac9afb-96d1-465c-bd18-3311659ec275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:35.361934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:35.364692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 30/64] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:35.382012 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-6dac9afb-96d1-465c-bd18-3311659ec275 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:35.383383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6dac9afb-96d1-465c-bd18-3311659ec275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:35.387979 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28fd5b-5252-4677-8199-0e90b5ad7ede None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:35.464243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 35/65] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:35.465853 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c41a5-0152-47d9-9fd5-0ac58628acf8 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:35.466359 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c41a5-0152-47d9-9fd5-0ac58628acf8 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:35.466861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40c41a5-0152-47d9-9fd5-0ac58628acf8 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:35.468662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 36/66] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:35.481460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d60a7-22b8-4b38-b9b3-46bffe72b9ec None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:35.482049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d60a7-22b8-4b38-b9b3-46bffe72b9ec None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:35.482518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d60a7-22b8-4b38-b9b3-46bffe72b9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:35.505050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-f95d60a7-22b8-4b38-b9b3-46bffe72b9ec None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:35.506556 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f95d60a7-22b8-4b38-b9b3-46bffe72b9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:35.518779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28fd5b-5252-4677-8199-0e90b5ad7ede None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:35.521553 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:35.560505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953b64c-c533-4a6d-a91c-0937b38328c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:35.561713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953b64c-c533-4a6d-a91c-0937b38328c3 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:35.562145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953b64c-c533-4a6d-a91c-0937b38328c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:35.562603 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953b64c-c533-4a6d-a91c-0937b38328c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:35.575258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953b64c-c533-4a6d-a91c-0937b38328c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:35.582338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 37/67] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-8953b64c-c533-4a6d-a91c-0937b38328c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:35.586002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:35.590540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:35.591298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:35.591298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:35.591298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:35.591298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:35.591298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:35.591298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:35.591298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:35.591298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:35.591298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:35.591298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:35.591298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:35.591298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:01:35.591863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 31/68] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:01:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:01:35.592332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953b64c-c533-4a6d-a91c-0937b38328c3 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:35.593508 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d60a7-22b8-4b38-b9b3-46bffe72b9ec None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:35.677522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:35.708678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953b64c-c533-4a6d-a91c-0937b38328c3 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:35.719356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d60a7-22b8-4b38-b9b3-46bffe72b9ec None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:35.747936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6a7ff-0920-4502-8581-f91a1ae1b564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:35.749100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6a7ff-0920-4502-8581-f91a1ae1b564 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:35.749594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6a7ff-0920-4502-8581-f91a1ae1b564 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:35.750047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6a7ff-0920-4502-8581-f91a1ae1b564 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:35.756749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0aac49-ca17-4fd7-bada-2de7631bc3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:35.757639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0aac49-ca17-4fd7-bada-2de7631bc3bd None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:35.758063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0aac49-ca17-4fd7-bada-2de7631bc3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:35.758514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0aac49-ca17-4fd7-bada-2de7631bc3bd None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:35.759445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0aac49-ca17-4fd7-bada-2de7631bc3bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:35.759942 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0aac49-ca17-4fd7-bada-2de7631bc3bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:35.761642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0aac49-ca17-4fd7-bada-2de7631bc3bd None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:35.762151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6a7ff-0920-4502-8581-f91a1ae1b564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-f5b6a7ff-0920-4502-8581-f91a1ae1b564 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:35.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:35.772110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:35.773293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:35.773293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:35.773293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:35.773293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:35.773293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:35.773293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:35.773293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:35.773293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:35.773293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:35.773293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:35.773293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:35.773293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:01:35.773293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 32/69] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:35.779345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6a7ff-0920-4502-8581-f91a1ae1b564 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:35.779562 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 38/70] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:35.780633 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0aac49-ca17-4fd7-bada-2de7631bc3bd None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:35.808550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:35.901525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0aac49-ca17-4fd7-bada-2de7631bc3bd None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:35.902218 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6a7ff-0920-4502-8581-f91a1ae1b564 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:35.931515 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:35.941035 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2006e-2dde-436f-bb09-e905f8f01bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:35.941723 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2006e-2dde-436f-bb09-e905f8f01bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:35.942809 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f4c3f-48bb-4d95-95d4-e91280f250ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:35.942917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2006e-2dde-436f-bb09-e905f8f01bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:35.942917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2006e-2dde-436f-bb09-e905f8f01bb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:35.943059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f4c3f-48bb-4d95-95d4-e91280f250ef None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:35.943059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f4c3f-48bb-4d95-95d4-e91280f250ef None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:35.943204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f4c3f-48bb-4d95-95d4-e91280f250ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:35.943361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b2006e-2dde-436f-bb09-e905f8f01bb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:35.943682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b2006e-2dde-436f-bb09-e905f8f01bb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:35.945131 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b2006e-2dde-436f-bb09-e905f8f01bb8 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:35.953604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f4c3f-48bb-4d95-95d4-e91280f250ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:35.954742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 39/71] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:01:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:35.956051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2006e-2dde-436f-bb09-e905f8f01bb8 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-e88f4c3f-48bb-4d95-95d4-e91280f250ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:35.962040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:35.962893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:35.963632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:35.963632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:35.963632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:35.963632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:35.963632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:35.963632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:35.963632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:35.963632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:35.963632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:35.963632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:35.963632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:01:35.963632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:01:35.963632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 33/72] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:01:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:35.964703 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d9be7-86b3-4877-9b19-222968abe2ad None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:35.964804 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d9be7-86b3-4877-9b19-222968abe2ad None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:35.965139 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d9be7-86b3-4877-9b19-222968abe2ad None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:35.966468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 34/73] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:35.969114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71bbe92-b9b1-41a8-a371-3e6df499761b None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:35.969114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71bbe92-b9b1-41a8-a371-3e6df499761b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:35.969114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71bbe92-b9b1-41a8-a371-3e6df499761b None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:35.969489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 35/74] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:35.970539 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71bbe92-b9b1-41a8-a371-3e6df499761b None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:36.064452 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:36.065384 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b2006e-2dde-436f-bb09-e905f8f01bb8 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:36.080890 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71bbe92-b9b1-41a8-a371-3e6df499761b None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:36.100631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c47dc-6677-4847-97df-b1666b1ccc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:36.101701 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c47dc-6677-4847-97df-b1666b1ccc42 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:36.101989 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c47dc-6677-4847-97df-b1666b1ccc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:36.102271 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c47dc-6677-4847-97df-b1666b1ccc42 None admin] PATH_INFO: `/v3/users/ede33ec052df4b3dab11128e172fd980` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:36.103563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c47dc-6677-4847-97df-b1666b1ccc42 None admin] RBAC: Authorizing `identity:get_user(user_id=ede33ec052df4b3dab11128e172fd980)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:36.105077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c47dc-6677-4847-97df-b1666b1ccc42 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:36.106742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 40/75] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:35 2022] GET /identity/v3/users/ede33ec052df4b3dab11128e172fd980 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:36.107486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c47dc-6677-4847-97df-b1666b1ccc42 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:36.118097 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d5222-cfef-4ad0-83cd-3a02bc91c807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:36.118995 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d5222-cfef-4ad0-83cd-3a02bc91c807 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:36.119288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d5222-cfef-4ad0-83cd-3a02bc91c807 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:36.119571 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d5222-cfef-4ad0-83cd-3a02bc91c807 None admin] PATH_INFO: `/v3/users/cdcce87a6587441884e93573b6bd6d5e` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:36.135583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d5222-cfef-4ad0-83cd-3a02bc91c807 None admin] RBAC: Authorizing `identity:get_user(user_id=cdcce87a6587441884e93573b6bd6d5e)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:36.137092 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d5222-cfef-4ad0-83cd-3a02bc91c807 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:36.138816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 36/76] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:35 2022] GET /identity/v3/users/cdcce87a6587441884e93573b6bd6d5e => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:36.140029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f468860-8981-4b60-97b4-2ba788b10e6f None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:36.140322 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f468860-8981-4b60-97b4-2ba788b10e6f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:36.140606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f468860-8981-4b60-97b4-2ba788b10e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:36.159345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-0f468860-8981-4b60-97b4-2ba788b10e6f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:36.160604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-0f468860-8981-4b60-97b4-2ba788b10e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:36.164508 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:36.209693 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c47dc-6677-4847-97df-b1666b1ccc42 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:36.230968 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 37/77] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:36.232262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b3f61-476a-416f-82cc-828d0bd066bf None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:36.232558 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b3f61-476a-416f-82cc-828d0bd066bf None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:36.232841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b3f61-476a-416f-82cc-828d0bd066bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:36.249520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-fa3b3f61-476a-416f-82cc-828d0bd066bf None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:36.250728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-fa3b3f61-476a-416f-82cc-828d0bd066bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:36.255710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26f809-9b2b-4c4c-9add-83a466b5ed75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:36.256389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26f809-9b2b-4c4c-9add-83a466b5ed75 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:36.256678 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26f809-9b2b-4c4c-9add-83a466b5ed75 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:36.256979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26f809-9b2b-4c4c-9add-83a466b5ed75 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:36.257755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26f809-9b2b-4c4c-9add-83a466b5ed75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:36.258068 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26f809-9b2b-4c4c-9add-83a466b5ed75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:36.259455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26f809-9b2b-4c4c-9add-83a466b5ed75 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:36.268606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 41/78] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:01:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:36.269489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26f809-9b2b-4c4c-9add-83a466b5ed75 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:36.324212 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 38/79] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:36.325025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b3f61-476a-416f-82cc-828d0bd066bf None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:36.327611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:36.376470 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26f809-9b2b-4c4c-9add-83a466b5ed75 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:36.394078 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:36.410825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6182e0-091b-4e97-a335-71fce1162847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:36.411487 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6182e0-091b-4e97-a335-71fce1162847 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:36.411779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6182e0-091b-4e97-a335-71fce1162847 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:36.412062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6182e0-091b-4e97-a335-71fce1162847 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:36.430138 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b3f61-476a-416f-82cc-828d0bd066bf None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:36.464304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53031d-881c-4773-9f4a-6978f01ea337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:36.464977 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53031d-881c-4773-9f4a-6978f01ea337 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:36.465279 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53031d-881c-4773-9f4a-6978f01ea337 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:36.465559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53031d-881c-4773-9f4a-6978f01ea337 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:36.479306 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6182e0-091b-4e97-a335-71fce1162847 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:36.480789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6182e0-091b-4e97-a335-71fce1162847 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:36.541033 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53031d-881c-4773-9f4a-6978f01ea337 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:36.542548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53031d-881c-4773-9f4a-6978f01ea337 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:36.544570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:36.587179 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 42/80] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:01:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:01:36.588547 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6182e0-091b-4e97-a335-71fce1162847 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:36.613576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:36.662678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 39/81] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:01:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:01:36.663529 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53031d-881c-4773-9f4a-6978f01ea337 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:36.697147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6182e0-091b-4e97-a335-71fce1162847 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:36.730177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f56a4d-119f-4b1e-9f3f-6b2441c4b0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:36.730977 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f56a4d-119f-4b1e-9f3f-6b2441c4b0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:36.731260 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f56a4d-119f-4b1e-9f3f-6b2441c4b0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:36.731537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f56a4d-119f-4b1e-9f3f-6b2441c4b0e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:36.742511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f56a4d-119f-4b1e-9f3f-6b2441c4b0e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-21f56a4d-119f-4b1e-9f3f-6b2441c4b0e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:36.750441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:36.751309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:36.752773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:36.752773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:36.752773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:36.752773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:36.752773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:36.752773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:36.752773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:36.752773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:36.752773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:36.752773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:36.752773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:36.752773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:01:36.752773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 43/82] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:36.753811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f56a4d-119f-4b1e-9f3f-6b2441c4b0e6 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:36.766375 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53031d-881c-4773-9f4a-6978f01ea337 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:36.768206 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:36.802326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cf986-a312-42ba-9612-cc4558d7cf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:36.803250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cf986-a312-42ba-9612-cc4558d7cf86 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:36.803944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cf986-a312-42ba-9612-cc4558d7cf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:36.804408 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7cf986-a312-42ba-9612-cc4558d7cf86 None admin] PATH_INFO: `/v3/users/cdcce87a6587441884e93573b6bd6d5e` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:36.805919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cf986-a312-42ba-9612-cc4558d7cf86 None admin] RBAC: Authorizing `identity:get_user(user_id=cdcce87a6587441884e93573b6bd6d5e)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:36.807408 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7cf986-a312-42ba-9612-cc4558d7cf86 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:36.809117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 40/83] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:36 2022] GET /identity/v3/users/cdcce87a6587441884e93573b6bd6d5e => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:36.809895 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cf986-a312-42ba-9612-cc4558d7cf86 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:36.833569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:36.857205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f56a4d-119f-4b1e-9f3f-6b2441c4b0e6 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:36.893044 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5c9cc-50ef-439d-a853-b247a6a1c750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:36.893936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5c9cc-50ef-439d-a853-b247a6a1c750 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:36.894221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5c9cc-50ef-439d-a853-b247a6a1c750 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:36.894501 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5c9cc-50ef-439d-a853-b247a6a1c750 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:36.895685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5c9cc-50ef-439d-a853-b247a6a1c750 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:36.897196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5c9cc-50ef-439d-a853-b247a6a1c750 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:36.898822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 44/84] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:01:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:36.899588 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5c9cc-50ef-439d-a853-b247a6a1c750 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:36.915393 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7cf986-a312-42ba-9612-cc4558d7cf86 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:36.949215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51e83a-a98b-4bcb-926d-0d13ca6e7b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:36.950086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51e83a-a98b-4bcb-926d-0d13ca6e7b38 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:36.950370 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51e83a-a98b-4bcb-926d-0d13ca6e7b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:36.950646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51e83a-a98b-4bcb-926d-0d13ca6e7b38 None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:36.951870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51e83a-a98b-4bcb-926d-0d13ca6e7b38 None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:36.953364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51e83a-a98b-4bcb-926d-0d13ca6e7b38 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:36.955101 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 41/85] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:36 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:36.955874 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51e83a-a98b-4bcb-926d-0d13ca6e7b38 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:36.996131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:36.999903 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5c9cc-50ef-439d-a853-b247a6a1c750 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:37.034130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2c4a8-5cd8-419c-b509-4019ae4152d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:37.035009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2c4a8-5cd8-419c-b509-4019ae4152d7 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:37.035300 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2c4a8-5cd8-419c-b509-4019ae4152d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:37.035586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2c4a8-5cd8-419c-b509-4019ae4152d7 None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:37.036796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2c4a8-5cd8-419c-b509-4019ae4152d7 None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:37.038317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c2c4a8-5cd8-419c-b509-4019ae4152d7 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:37.045090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 45/86] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:36 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:37.045090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2c4a8-5cd8-419c-b509-4019ae4152d7 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:37.057711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51e83a-a98b-4bcb-926d-0d13ca6e7b38 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:37.059227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:37.092421 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797c5e9-5a7a-4c4d-a5de-f8752fe1d160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:37.093136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797c5e9-5a7a-4c4d-a5de-f8752fe1d160 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:37.093432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797c5e9-5a7a-4c4d-a5de-f8752fe1d160 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:37.093715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797c5e9-5a7a-4c4d-a5de-f8752fe1d160 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:37.094557 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797c5e9-5a7a-4c4d-a5de-f8752fe1d160 None admin] RBAC: Adding query filter params (name=member) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:37.094877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797c5e9-5a7a-4c4d-a5de-f8752fe1d160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:37.096981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797c5e9-5a7a-4c4d-a5de-f8752fe1d160 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:37.105285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 42/87] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:01:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:37.106063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797c5e9-5a7a-4c4d-a5de-f8752fe1d160 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:37.149149 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2c4a8-5cd8-419c-b509-4019ae4152d7 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:37.184591 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcb461-8180-46cc-915e-5176c181d479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:37.185285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcb461-8180-46cc-915e-5176c181d479 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:37.185574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcb461-8180-46cc-915e-5176c181d479 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:37.185851 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcb461-8180-46cc-915e-5176c181d479 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:37.207093 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:37.210423 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797c5e9-5a7a-4c4d-a5de-f8752fe1d160 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:37.247331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fc2a1-1a35-475a-af2d-2f87eb894d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:37.248006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fc2a1-1a35-475a-af2d-2f87eb894d8d None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:37.248293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fc2a1-1a35-475a-af2d-2f87eb894d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:37.248599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fc2a1-1a35-475a-af2d-2f87eb894d8d None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:37.249382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168fc2a1-1a35-475a-af2d-2f87eb894d8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:37.250827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168fc2a1-1a35-475a-af2d-2f87eb894d8d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:37.262538 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fcb461-8180-46cc-915e-5176c181d479 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:37.264046 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fcb461-8180-46cc-915e-5176c181d479 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:37.279088 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:37.319544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 43/88] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:01:37 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:01:37.320425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fc2a1-1a35-475a-af2d-2f87eb894d8d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:37.371051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 46/89] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:01:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:01:37.372189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcb461-8180-46cc-915e-5176c181d479 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:37.453628 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fc2a1-1a35-475a-af2d-2f87eb894d8d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:37.489246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcb461-8180-46cc-915e-5176c181d479 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:37.503307 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6e595-d47f-4945-af20-ea53f21e045f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:37.504015 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6e595-d47f-4945-af20-ea53f21e045f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:37.504310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6e595-d47f-4945-af20-ea53f21e045f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:37.504594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6e595-d47f-4945-af20-ea53f21e045f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:37.505766 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6e595-d47f-4945-af20-ea53f21e045f None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=2710654d490f43b0adadfd7a62df84bd, user.id=ede33ec052df4b3dab11128e172fd980) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:37.506094 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6e595-d47f-4945-af20-ea53f21e045f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:37.506560 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:37.508345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6e595-d47f-4945-af20-ea53f21e045f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:37.514772 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 44/90] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:01:37 2022] GET /identity/v3/role_assignments?user.id=ede33ec052df4b3dab11128e172fd980&scope.project.id=2710654d490f43b0adadfd7a62df84bd&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:37.515688 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6e595-d47f-4945-af20-ea53f21e045f None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:37.531084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88da908-4992-45ca-a47e-437ccda685c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:37.531768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88da908-4992-45ca-a47e-437ccda685c0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:37.532077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88da908-4992-45ca-a47e-437ccda685c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:37.532453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88da908-4992-45ca-a47e-437ccda685c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:37.534139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88da908-4992-45ca-a47e-437ccda685c0 None admin] RBAC: Adding query filter params (role.id=c91413ec3feb454ca7186eb52a57ed96, scope.project.id=2710654d490f43b0adadfd7a62df84bd, user.id=cdcce87a6587441884e93573b6bd6d5e) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:37.534425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88da908-4992-45ca-a47e-437ccda685c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:37.536007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88da908-4992-45ca-a47e-437ccda685c0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:37.547522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 47/91] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:01:37 2022] GET /identity/v3/role_assignments?user.id=cdcce87a6587441884e93573b6bd6d5e&scope.project.id=2710654d490f43b0adadfd7a62df84bd&role.id=c91413ec3feb454ca7186eb52a57ed96 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:37.548265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88da908-4992-45ca-a47e-437ccda685c0 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:37.551594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:37.656540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6e595-d47f-4945-af20-ea53f21e045f None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:37.676519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88da908-4992-45ca-a47e-437ccda685c0 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:37.700693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1a5fb-6fbf-46da-9f7b-8259c2394421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:37.701669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1a5fb-6fbf-46da-9f7b-8259c2394421 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:37.702045 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1a5fb-6fbf-46da-9f7b-8259c2394421 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:37.702235 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1a5fb-6fbf-46da-9f7b-8259c2394421 None admin] PATH_INFO: `/v3/users/cdcce87a6587441884e93573b6bd6d5e` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:37.703609 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a1a5fb-6fbf-46da-9f7b-8259c2394421 None admin] RBAC: Authorizing `identity:get_user(user_id=cdcce87a6587441884e93573b6bd6d5e)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:37.705463 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a1a5fb-6fbf-46da-9f7b-8259c2394421 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:37.707544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 45/92] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:37 2022] GET /identity/v3/users/cdcce87a6587441884e93573b6bd6d5e => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:37.716704 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1a5fb-6fbf-46da-9f7b-8259c2394421 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:37.730746 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1b271-5460-4e7b-b409-e8f533b835c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:37.731653 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1b271-5460-4e7b-b409-e8f533b835c0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:37.732057 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1b271-5460-4e7b-b409-e8f533b835c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:37.732357 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1b271-5460-4e7b-b409-e8f533b835c0 None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:37.733715 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1b271-5460-4e7b-b409-e8f533b835c0 None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:37.735284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc1b271-5460-4e7b-b409-e8f533b835c0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:37.738298 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 48/93] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:37 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:37.744382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1b271-5460-4e7b-b409-e8f533b835c0 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:37.803542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:37.840383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:37.861507 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1a5fb-6fbf-46da-9f7b-8259c2394421 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:37.876010 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1b271-5460-4e7b-b409-e8f533b835c0 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:37.910877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77730ba-1aa7-43fc-8aed-87538fdd1855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:37.911548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77730ba-1aa7-43fc-8aed-87538fdd1855 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:37.911839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77730ba-1aa7-43fc-8aed-87538fdd1855 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:37.912121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77730ba-1aa7-43fc-8aed-87538fdd1855 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:37.930310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7559560-3a8e-4c0d-91a0-82bed984e842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:37.930986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7559560-3a8e-4c0d-91a0-82bed984e842 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:37.931272 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7559560-3a8e-4c0d-91a0-82bed984e842 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:37.931551 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7559560-3a8e-4c0d-91a0-82bed984e842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:37.932399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7559560-3a8e-4c0d-91a0-82bed984e842 None admin] RBAC: Adding query filter params (role.id=d241ebe8f5654f46a5a92be6f6b0cc36, scope.project.id=2710654d490f43b0adadfd7a62df84bd, user.id=cdcce87a6587441884e93573b6bd6d5e) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:37.932714 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7559560-3a8e-4c0d-91a0-82bed984e842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:37.934211 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7559560-3a8e-4c0d-91a0-82bed984e842 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:37.946439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 49/94] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:01:37 2022] GET /identity/v3/role_assignments?user.id=cdcce87a6587441884e93573b6bd6d5e&scope.project.id=2710654d490f43b0adadfd7a62df84bd&role.id=d241ebe8f5654f46a5a92be6f6b0cc36 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:38.019934 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77730ba-1aa7-43fc-8aed-87538fdd1855 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:38.021366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77730ba-1aa7-43fc-8aed-87538fdd1855 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:38.129971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:38.144745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 46/95] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:01:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:01:38.151142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7559560-3a8e-4c0d-91a0-82bed984e842 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:38.319490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7559560-3a8e-4c0d-91a0-82bed984e842 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:38.331746 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:38.357569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd97c8-cf94-4c5e-acf7-83c092f5dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:38.358575 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd97c8-cf94-4c5e-acf7-83c092f5dc0b None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:38.358863 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd97c8-cf94-4c5e-acf7-83c092f5dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:38.359144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd97c8-cf94-4c5e-acf7-83c092f5dc0b None admin] PATH_INFO: `/v3/projects/3ab69a8ea9d94395a6b9ae0110293214` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:38.360453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd97c8-cf94-4c5e-acf7-83c092f5dc0b None admin] RBAC: Authorizing `identity:get_project(project_id=3ab69a8ea9d94395a6b9ae0110293214)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:38.362550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd97c8-cf94-4c5e-acf7-83c092f5dc0b None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:38.369253 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 50/96] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:38 2022] GET /identity/v3/projects/3ab69a8ea9d94395a6b9ae0110293214 => generated 312 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:38.370694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77730ba-1aa7-43fc-8aed-87538fdd1855 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:38.477542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77730ba-1aa7-43fc-8aed-87538fdd1855 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:38.520737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf911734-aa97-45bb-b4a9-fa957f7d795a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:38.521432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf911734-aa97-45bb-b4a9-fa957f7d795a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:38.521720 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf911734-aa97-45bb-b4a9-fa957f7d795a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:38.522000 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf911734-aa97-45bb-b4a9-fa957f7d795a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:38.525193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf911734-aa97-45bb-b4a9-fa957f7d795a None admin] RBAC: Adding query filter params (role.id=c91413ec3feb454ca7186eb52a57ed96, scope.project.id=3ab69a8ea9d94395a6b9ae0110293214, user.id=cdcce87a6587441884e93573b6bd6d5e) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:38.525193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf911734-aa97-45bb-b4a9-fa957f7d795a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:38.525193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf911734-aa97-45bb-b4a9-fa957f7d795a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:38.534271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 47/97] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:01:38 2022] GET /identity/v3/role_assignments?user.id=cdcce87a6587441884e93573b6bd6d5e&scope.project.id=3ab69a8ea9d94395a6b9ae0110293214&role.id=c91413ec3feb454ca7186eb52a57ed96 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:40.425281 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff56cd-92ae-416c-8c2c-882793d9259b None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:40.426649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff56cd-92ae-416c-8c2c-882793d9259b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:40.427163 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff56cd-92ae-416c-8c2c-882793d9259b None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:40.429253 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 51/98] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:40.435556 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59b2d4-f4ff-43a3-991b-1ef5bdd7f60c None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:40.436481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59b2d4-f4ff-43a3-991b-1ef5bdd7f60c None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:40.437035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59b2d4-f4ff-43a3-991b-1ef5bdd7f60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:40.455735 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-ce59b2d4-f4ff-43a3-991b-1ef5bdd7f60c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:40.457391 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ce59b2d4-f4ff-43a3-991b-1ef5bdd7f60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:40.503599 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f867b-1e6a-451e-ab11-328336bd2dfd None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:40.504258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f867b-1e6a-451e-ab11-328336bd2dfd None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:40.504754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f867b-1e6a-451e-ab11-328336bd2dfd None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:40.506385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 52/99] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:01:40.512100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66e8bd-9ac8-46d5-8951-cea3ae3dc6a3 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:40.512649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66e8bd-9ac8-46d5-8951-cea3ae3dc6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:40.513153 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66e8bd-9ac8-46d5-8951-cea3ae3dc6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:40.536733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-ca66e8bd-9ac8-46d5-8951-cea3ae3dc6a3 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:40.538317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ca66e8bd-9ac8-46d5-8951-cea3ae3dc6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:40.562735 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:40.605281 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 48/100] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:40.623443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59b2d4-f4ff-43a3-991b-1ef5bdd7f60c None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:40.717896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 53/101] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:40.754164 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66e8bd-9ac8-46d5-8951-cea3ae3dc6a3 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:40.776309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59b2d4-f4ff-43a3-991b-1ef5bdd7f60c None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:40.861998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3a626-23c1-4f62-a844-a245e53c6b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:40.863073 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3a626-23c1-4f62-a844-a245e53c6b8f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:40.863536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3a626-23c1-4f62-a844-a245e53c6b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:40.863962 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3a626-23c1-4f62-a844-a245e53c6b8f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:40.884334 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3a626-23c1-4f62-a844-a245e53c6b8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-57f3a626-23c1-4f62-a844-a245e53c6b8f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:40.893439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:40.894301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:40.895050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:40.895050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:40.895050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:40.895050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:40.895050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:40.895050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:40.895050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:40.895050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:40.895050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:40.895050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:40.895050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:40.895050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:01:40.895640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 49/102] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:01:40.902238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf864d-ffd1-4fa8-a4ab-c13d8902d029 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:40.902556 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf864d-ffd1-4fa8-a4ab-c13d8902d029 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:40.902849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf864d-ffd1-4fa8-a4ab-c13d8902d029 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:40.904291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 50/103] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:40.906463 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf864d-ffd1-4fa8-a4ab-c13d8902d029 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:40.962914 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:41.029919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf864d-ffd1-4fa8-a4ab-c13d8902d029 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:41.051855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66e8bd-9ac8-46d5-8951-cea3ae3dc6a3 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:41.064707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:41.072331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837ce16-0d72-4b8b-af59-2b8699ffeed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:41.073456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837ce16-0d72-4b8b-af59-2b8699ffeed4 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:41.073960 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837ce16-0d72-4b8b-af59-2b8699ffeed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:41.074449 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837ce16-0d72-4b8b-af59-2b8699ffeed4 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:41.075887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837ce16-0d72-4b8b-af59-2b8699ffeed4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:41.076632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837ce16-0d72-4b8b-af59-2b8699ffeed4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:41.078753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837ce16-0d72-4b8b-af59-2b8699ffeed4 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:41.090481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 51/104] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:01:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:41.094696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba62d55-f58d-437c-b43d-9f9d92f4df4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:41.094696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba62d55-f58d-437c-b43d-9f9d92f4df4e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:41.094696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba62d55-f58d-437c-b43d-9f9d92f4df4e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:41.094696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba62d55-f58d-437c-b43d-9f9d92f4df4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:41.097434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeca0a3-2076-416d-b497-fa0bcb3d07f3 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:41.097911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeca0a3-2076-416d-b497-fa0bcb3d07f3 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:41.098363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeca0a3-2076-416d-b497-fa0bcb3d07f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:41.105679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba62d55-f58d-437c-b43d-9f9d92f4df4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-3ba62d55-f58d-437c-b43d-9f9d92f4df4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:41.115559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:41.116450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:41.117307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:41.117307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:41.117307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:41.117307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:41.117307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:41.117307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:41.117307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:41.117307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:41.117307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:41.117307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:41.117307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:41.117307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:01:41.117307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 54/105] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:01:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:01:41.117925 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba62d55-f58d-437c-b43d-9f9d92f4df4e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:41.122760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-8aeca0a3-2076-416d-b497-fa0bcb3d07f3 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:41.124509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-8aeca0a3-2076-416d-b497-fa0bcb3d07f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:41.275255 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 52/106] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:41.276434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeca0a3-2076-416d-b497-fa0bcb3d07f3 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:41.276536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba62d55-f58d-437c-b43d-9f9d92f4df4e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:41.319225 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:41.328625 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d7029f-6740-49a5-a224-49d55598e09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:41.330542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7029f-6740-49a5-a224-49d55598e09b None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:41.330833 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7029f-6740-49a5-a224-49d55598e09b None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:41.331117 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d7029f-6740-49a5-a224-49d55598e09b None admin] PATH_INFO: `/v3/users/cdcce87a6587441884e93573b6bd6d5e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:41.333027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7029f-6740-49a5-a224-49d55598e09b None admin] RBAC: Authorizing `identity:get_user(user_id=cdcce87a6587441884e93573b6bd6d5e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:41.334749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d7029f-6740-49a5-a224-49d55598e09b None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:41.337073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 55/107] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:41 2022] GET /identity/v3/users/cdcce87a6587441884e93573b6bd6d5e => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:41.338467 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629961be-5ba8-417c-bba1-e9062c4d844b None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:41.338759 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629961be-5ba8-417c-bba1-e9062c4d844b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:41.339044 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629961be-5ba8-417c-bba1-e9062c4d844b None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:41.340509 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 56/108] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:41.344878 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629961be-5ba8-417c-bba1-e9062c4d844b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:41.400227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:41.410742 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeca0a3-2076-416d-b497-fa0bcb3d07f3 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:41.457493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbb096-8902-4f55-a090-9ebc5629d476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:41.458244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbb096-8902-4f55-a090-9ebc5629d476 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:41.458522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbb096-8902-4f55-a090-9ebc5629d476 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:41.458832 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbb096-8902-4f55-a090-9ebc5629d476 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:41.460187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbb096-8902-4f55-a090-9ebc5629d476 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:41.460508 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbb096-8902-4f55-a090-9ebc5629d476 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:41.462002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbb096-8902-4f55-a090-9ebc5629d476 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:41.478351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 53/109] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:41.478351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbb096-8902-4f55-a090-9ebc5629d476 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:41.503751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629961be-5ba8-417c-bba1-e9062c4d844b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:41.559403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a944638-af83-4c9a-acd7-e6e8652337c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:41.561376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a944638-af83-4c9a-acd7-e6e8652337c4 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:41.561969 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a944638-af83-4c9a-acd7-e6e8652337c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:41.562574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a944638-af83-4c9a-acd7-e6e8652337c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:41.565626 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a944638-af83-4c9a-acd7-e6e8652337c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:41.567113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a944638-af83-4c9a-acd7-e6e8652337c4 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:41.573756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 57/110] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:01:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:41.573756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c5cce-935d-41dd-a58d-5411ff5d439a None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:41.573756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c5cce-935d-41dd-a58d-5411ff5d439a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:41.573756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c5cce-935d-41dd-a58d-5411ff5d439a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:41.598404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-746c5cce-935d-41dd-a58d-5411ff5d439a None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:41.600403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-746c5cce-935d-41dd-a58d-5411ff5d439a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:41.612539 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:41.621267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbb096-8902-4f55-a090-9ebc5629d476 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:41.667391 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389956f-c83b-4ebb-a1dd-591ab92efe14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:41.668810 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389956f-c83b-4ebb-a1dd-591ab92efe14 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:41.669330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389956f-c83b-4ebb-a1dd-591ab92efe14 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:41.669873 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389956f-c83b-4ebb-a1dd-591ab92efe14 None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:41.671511 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389956f-c83b-4ebb-a1dd-591ab92efe14 None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:41.674020 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389956f-c83b-4ebb-a1dd-591ab92efe14 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:41.676638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 54/111] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:41 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:41.677664 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389956f-c83b-4ebb-a1dd-591ab92efe14 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:41.690518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 58/112] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:41.691359 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c5cce-935d-41dd-a58d-5411ff5d439a None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:41.727023 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:41.793142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c5cce-935d-41dd-a58d-5411ff5d439a None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:41.795531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389956f-c83b-4ebb-a1dd-591ab92efe14 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:41.828396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193f364-34a5-420b-af2e-60721c1f1ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:41.829024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193f364-34a5-420b-af2e-60721c1f1ac3 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:41.829392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193f364-34a5-420b-af2e-60721c1f1ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:41.829683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193f364-34a5-420b-af2e-60721c1f1ac3 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:41.830631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0193f364-34a5-420b-af2e-60721c1f1ac3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:41.831089 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f42df-071b-47ef-aad8-b935cb9a275f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:41.831996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f42df-071b-47ef-aad8-b935cb9a275f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:41.832136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0193f364-34a5-420b-af2e-60721c1f1ac3 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:41.832289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f42df-071b-47ef-aad8-b935cb9a275f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:41.832572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f42df-071b-47ef-aad8-b935cb9a275f None admin] PATH_INFO: `/v3/users/ede33ec052df4b3dab11128e172fd980` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:41.834047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f42df-071b-47ef-aad8-b935cb9a275f None admin] RBAC: Authorizing `identity:get_user(user_id=ede33ec052df4b3dab11128e172fd980)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:41.835537 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f42df-071b-47ef-aad8-b935cb9a275f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:41.837344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 55/113] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:41 2022] GET /identity/v3/users/ede33ec052df4b3dab11128e172fd980 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:41.838182 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f42df-071b-47ef-aad8-b935cb9a275f None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:41.873979 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:41.939541 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 59/114] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jun 15 10:01:41 2022] POST /identity/v3/projects => generated 302 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:01:41.941198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2bc2c-ec61-4bdd-8908-b743a49929a4 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:41.941562 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2bc2c-ec61-4bdd-8908-b743a49929a4 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:41.941848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2bc2c-ec61-4bdd-8908-b743a49929a4 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:41.943215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 60/115] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:41.947631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a50ab-b24b-4a9e-bee6-20d81a9f50ea None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:41.947631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a50ab-b24b-4a9e-bee6-20d81a9f50ea None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:41.947631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a50ab-b24b-4a9e-bee6-20d81a9f50ea None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:41.947631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 61/116] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:41.947631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a50ab-b24b-4a9e-bee6-20d81a9f50ea None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:41.954111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f42df-071b-47ef-aad8-b935cb9a275f None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:41.995806 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9900404-fcbc-4fa9-8081-1c4b34ab8c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:41.996727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9900404-fcbc-4fa9-8081-1c4b34ab8c70 None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:41.997051 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9900404-fcbc-4fa9-8081-1c4b34ab8c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:41.997340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9900404-fcbc-4fa9-8081-1c4b34ab8c70 None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd/users/cdcce87a6587441884e93573b6bd6d5e/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:41.999521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9900404-fcbc-4fa9-8081-1c4b34ab8c70 None admin] RBAC: Authorizing `identity:create_grant(project_id=2710654d490f43b0adadfd7a62df84bd, user_id=cdcce87a6587441884e93573b6bd6d5e, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:42.001916 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9900404-fcbc-4fa9-8081-1c4b34ab8c70 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:42.046730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:42.064555 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a50ab-b24b-4a9e-bee6-20d81a9f50ea None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:42.102166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5972e-394c-482d-a7ff-3ebce091edbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:42.102899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5972e-394c-482d-a7ff-3ebce091edbd None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:42.103032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 56/117] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:01:41 2022] PUT /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd/users/cdcce87a6587441884e93573b6bd6d5e/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:01:42.103175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5972e-394c-482d-a7ff-3ebce091edbd None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:42.103445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5972e-394c-482d-a7ff-3ebce091edbd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:42.104113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9900404-fcbc-4fa9-8081-1c4b34ab8c70 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:42.116877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5972e-394c-482d-a7ff-3ebce091edbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-96f5972e-394c-482d-a7ff-3ebce091edbd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:42.125081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:42.126113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:42.127464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:42.127464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:42.127464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:42.127464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:42.127464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:42.127464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:42.127464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:42.127464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:42.127464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:42.127464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:42.127464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:01:42.127464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:01:42.127464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 62/118] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:42.128614 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663bd13-0689-4790-b051-a8bfddc6a23b None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:42.128727 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663bd13-0689-4790-b051-a8bfddc6a23b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:42.128984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663bd13-0689-4790-b051-a8bfddc6a23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:42.150720 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-5663bd13-0689-4790-b051-a8bfddc6a23b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:42.151960 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5663bd13-0689-4790-b051-a8bfddc6a23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:42.227169 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:42.239151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9900404-fcbc-4fa9-8081-1c4b34ab8c70 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:42.253949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 63/119] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:42.255303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc128be-12fd-477d-afd6-ca0fd4fad5bb None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:42.255601 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc128be-12fd-477d-afd6-ca0fd4fad5bb None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:42.255953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc128be-12fd-477d-afd6-ca0fd4fad5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:42.281058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-dbc128be-12fd-477d-afd6-ca0fd4fad5bb None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:42.282017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-dbc128be-12fd-477d-afd6-ca0fd4fad5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:42.304615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e287d66-7d37-441d-a130-3dbcd33ef9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:42.305571 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e287d66-7d37-441d-a130-3dbcd33ef9ca None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:42.305862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e287d66-7d37-441d-a130-3dbcd33ef9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:42.306378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e287d66-7d37-441d-a130-3dbcd33ef9ca None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:42.310429 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e287d66-7d37-441d-a130-3dbcd33ef9ca None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:42.312435 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e287d66-7d37-441d-a130-3dbcd33ef9ca None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:42.315959 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 57/120] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:42 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:42.316530 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e287d66-7d37-441d-a130-3dbcd33ef9ca None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:42.371226 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 64/121] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:42.372011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc128be-12fd-477d-afd6-ca0fd4fad5bb None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:42.381274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:42.453481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e287d66-7d37-441d-a130-3dbcd33ef9ca None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:42.489410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc128be-12fd-477d-afd6-ca0fd4fad5bb None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:42.497213 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fdcd7-9d54-4ca1-a233-36f4796f75f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:42.498595 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fdcd7-9d54-4ca1-a233-36f4796f75f2 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:42.499042 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fdcd7-9d54-4ca1-a233-36f4796f75f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:42.499475 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fdcd7-9d54-4ca1-a233-36f4796f75f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:42.500390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fdcd7-9d54-4ca1-a233-36f4796f75f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:42.500945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fdcd7-9d54-4ca1-a233-36f4796f75f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:42.503153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fdcd7-9d54-4ca1-a233-36f4796f75f2 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:42.513879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 58/122] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:01:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:42.515116 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fdcd7-9d54-4ca1-a233-36f4796f75f2 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:42.535315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6816449-ae69-4d46-8c30-532eff92051e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:42.536157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6816449-ae69-4d46-8c30-532eff92051e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:42.536469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6816449-ae69-4d46-8c30-532eff92051e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:42.536896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6816449-ae69-4d46-8c30-532eff92051e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:42.553734 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6816449-ae69-4d46-8c30-532eff92051e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:42.553936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-a6816449-ae69-4d46-8c30-532eff92051e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:42.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:42.569107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:42.570024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:42.570024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:42.570024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:42.570024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:42.570024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:42.570024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:42.570024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:42.570024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:42.570024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:42.570024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:42.570024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:01:42.570024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:01:42.570024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 65/123] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:01:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:42.570024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6816449-ae69-4d46-8c30-532eff92051e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:42.647578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fdcd7-9d54-4ca1-a233-36f4796f75f2 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:42.669088 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:42.695245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331a242-1650-4741-8656-e0f667e8d202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:42.696290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331a242-1650-4741-8656-e0f667e8d202 None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:42.696581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331a242-1650-4741-8656-e0f667e8d202 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:42.696865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331a242-1650-4741-8656-e0f667e8d202 None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd/users/ede33ec052df4b3dab11128e172fd980/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:42.697956 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6816449-ae69-4d46-8c30-532eff92051e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:42.698946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331a242-1650-4741-8656-e0f667e8d202 None admin] RBAC: Authorizing `identity:create_grant(project_id=2710654d490f43b0adadfd7a62df84bd, user_id=ede33ec052df4b3dab11128e172fd980, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:42.701224 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331a242-1650-4741-8656-e0f667e8d202 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:42.738623 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8c805-eec6-4d54-b3e1-00e6486ba9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:42.739661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8c805-eec6-4d54-b3e1-00e6486ba9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:42.740085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8c805-eec6-4d54-b3e1-00e6486ba9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:42.740558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8c805-eec6-4d54-b3e1-00e6486ba9b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:42.753802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8c805-eec6-4d54-b3e1-00e6486ba9b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-cba8c805-eec6-4d54-b3e1-00e6486ba9b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:42.762024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:42.763008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:42.763885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:42.763885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:42.763885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:42.763885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:42.763885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:42.763885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:42.763885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:42.763885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:42.763885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:42.763885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:42.763885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:42.763885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:01:42.766022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 66/124] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:42.767756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8c805-eec6-4d54-b3e1-00e6486ba9b0 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:42.789470 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 59/125] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:01:42 2022] PUT /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd/users/ede33ec052df4b3dab11128e172fd980/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:01:42.794722 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331a242-1650-4741-8656-e0f667e8d202 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:42.985688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8c805-eec6-4d54-b3e1-00e6486ba9b0 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:42.999869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:43.027607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331a242-1650-4741-8656-e0f667e8d202 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:43.029793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:43.044493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aab919-6f69-4493-8aa7-722eab7bfbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:43.045716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aab919-6f69-4493-8aa7-722eab7bfbdc None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:43.046148 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aab919-6f69-4493-8aa7-722eab7bfbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:43.046589 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aab919-6f69-4493-8aa7-722eab7bfbdc None admin] PATH_INFO: `/v3/users/87dd1617d8f84685bd7cbcf2d4ac83ae` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:43.062604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aab919-6f69-4493-8aa7-722eab7bfbdc None admin] RBAC: Authorizing `identity:get_user(user_id=87dd1617d8f84685bd7cbcf2d4ac83ae)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:43.064563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aab919-6f69-4493-8aa7-722eab7bfbdc None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:43.066719 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 67/126] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:42 2022] GET /identity/v3/users/87dd1617d8f84685bd7cbcf2d4ac83ae => generated 289 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:43.068027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aab919-6f69-4493-8aa7-722eab7bfbdc None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:43.069673 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5df318-f26f-49c9-969d-f686adc81e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:43.070458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df318-f26f-49c9-969d-f686adc81e8c None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:43.070846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df318-f26f-49c9-969d-f686adc81e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:43.071302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df318-f26f-49c9-969d-f686adc81e8c None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:43.072182 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5df318-f26f-49c9-969d-f686adc81e8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:43.073413 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5df318-f26f-49c9-969d-f686adc81e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:43.074140 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5df318-f26f-49c9-969d-f686adc81e8c None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:43.083662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 60/127] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:01:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:43.084458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5df318-f26f-49c9-969d-f686adc81e8c None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:43.180705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aab919-6f69-4493-8aa7-722eab7bfbdc None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:43.218095 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156e0b8-7f59-4f7d-a3da-ff919b9a516c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:43.219027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156e0b8-7f59-4f7d-a3da-ff919b9a516c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:43.219461 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156e0b8-7f59-4f7d-a3da-ff919b9a516c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:43.219953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156e0b8-7f59-4f7d-a3da-ff919b9a516c None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:43.220888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156e0b8-7f59-4f7d-a3da-ff919b9a516c None admin] RBAC: Adding query filter params (name=member) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:43.221395 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156e0b8-7f59-4f7d-a3da-ff919b9a516c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:43.222952 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156e0b8-7f59-4f7d-a3da-ff919b9a516c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:43.232342 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 68/128] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:01:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:43.233686 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156e0b8-7f59-4f7d-a3da-ff919b9a516c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:43.254822 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5df318-f26f-49c9-969d-f686adc81e8c None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:43.286822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:43.317793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b1725-23cd-4965-a230-24a1f6131e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:43.318652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b1725-23cd-4965-a230-24a1f6131e4a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:43.319049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b1725-23cd-4965-a230-24a1f6131e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:43.319440 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b1725-23cd-4965-a230-24a1f6131e4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:43.350441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156e0b8-7f59-4f7d-a3da-ff919b9a516c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:43.350809 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:43.397217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3afbd4-5558-44ca-818b-03bf49e5e801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:43.398443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3afbd4-5558-44ca-818b-03bf49e5e801 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:43.398874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3afbd4-5558-44ca-818b-03bf49e5e801 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:43.399300 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3afbd4-5558-44ca-818b-03bf49e5e801 None admin] PATH_INFO: `/v3/users/cdcce87a6587441884e93573b6bd6d5e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:43.400813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3afbd4-5558-44ca-818b-03bf49e5e801 None admin] RBAC: Authorizing `identity:get_user(user_id=cdcce87a6587441884e93573b6bd6d5e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:43.402459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3afbd4-5558-44ca-818b-03bf49e5e801 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:43.404293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 69/129] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:43 2022] GET /identity/v3/users/cdcce87a6587441884e93573b6bd6d5e => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:43.405496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3afbd4-5558-44ca-818b-03bf49e5e801 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:43.457132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b1725-23cd-4965-a230-24a1f6131e4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:43.458807 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b1725-23cd-4965-a230-24a1f6131e4a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:43.518461 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3afbd4-5558-44ca-818b-03bf49e5e801 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:43.557355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:43.588478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f336f4a-d017-44ef-8aad-080f68517aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:43.589402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f336f4a-d017-44ef-8aad-080f68517aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:43.589690 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f336f4a-d017-44ef-8aad-080f68517aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:43.589972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f336f4a-d017-44ef-8aad-080f68517aa8 None admin] PATH_INFO: `/v3/users/cdcce87a6587441884e93573b6bd6d5e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:43.591286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f336f4a-d017-44ef-8aad-080f68517aa8 None admin] RBAC: Authorizing `identity:get_user(user_id=cdcce87a6587441884e93573b6bd6d5e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:43.592817 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f336f4a-d017-44ef-8aad-080f68517aa8 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:43.594508 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 70/130] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:43 2022] GET /identity/v3/users/cdcce87a6587441884e93573b6bd6d5e => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:43.599822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f336f4a-d017-44ef-8aad-080f68517aa8 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:43.656604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 61/131] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:01:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:01:43.658105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b1725-23cd-4965-a230-24a1f6131e4a None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:43.755770 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:43.811796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f336f4a-d017-44ef-8aad-080f68517aa8 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:43.866379 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b1725-23cd-4965-a230-24a1f6131e4a None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:43.888849 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57e9b0-7b52-4b78-8b26-9d4e5652b461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:43.889756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57e9b0-7b52-4b78-8b26-9d4e5652b461 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:43.890041 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57e9b0-7b52-4b78-8b26-9d4e5652b461 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:43.890317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd57e9b0-7b52-4b78-8b26-9d4e5652b461 None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:43.893727 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd57e9b0-7b52-4b78-8b26-9d4e5652b461 None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:43.896556 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd57e9b0-7b52-4b78-8b26-9d4e5652b461 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:43.898600 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 71/132] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:43 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:43.899199 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57e9b0-7b52-4b78-8b26-9d4e5652b461 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:43.934626 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4bea3-d503-4f4b-bd74-89b1141491f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:43.935527 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4bea3-d503-4f4b-bd74-89b1141491f4 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:43.935828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4bea3-d503-4f4b-bd74-89b1141491f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:43.936102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4bea3-d503-4f4b-bd74-89b1141491f4 None admin] PATH_INFO: `/v3/projects/3ab69a8ea9d94395a6b9ae0110293214` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:43.940904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4bea3-d503-4f4b-bd74-89b1141491f4 None admin] RBAC: Authorizing `identity:get_project(project_id=3ab69a8ea9d94395a6b9ae0110293214)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:43.943633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4bea3-d503-4f4b-bd74-89b1141491f4 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:43.945431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 62/133] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:43 2022] GET /identity/v3/projects/3ab69a8ea9d94395a6b9ae0110293214 => generated 312 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:43.948760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4bea3-d503-4f4b-bd74-89b1141491f4 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:44.021349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:44.083801 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd57e9b0-7b52-4b78-8b26-9d4e5652b461 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:44.120476 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0ba04-d7bf-48ba-b304-edef1eda85f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:44.121391 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0ba04-d7bf-48ba-b304-edef1eda85f6 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:44.121711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0ba04-d7bf-48ba-b304-edef1eda85f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:44.122677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0ba04-d7bf-48ba-b304-edef1eda85f6 None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:44.123304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0ba04-d7bf-48ba-b304-edef1eda85f6 None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:44.125043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de0ba04-d7bf-48ba-b304-edef1eda85f6 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:44.132740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 72/134] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:43 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:44.133667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0ba04-d7bf-48ba-b304-edef1eda85f6 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:44.208886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4bea3-d503-4f4b-bd74-89b1141491f4 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:44.246693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:44.247628 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0ba04-d7bf-48ba-b304-edef1eda85f6 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:44.267533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:44.287619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202167ee-f065-4821-8af3-f59d265b5bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:44.288598 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202167ee-f065-4821-8af3-f59d265b5bd7 None admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:44.288892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202167ee-f065-4821-8af3-f59d265b5bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:44.289212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202167ee-f065-4821-8af3-f59d265b5bd7 None admin] PATH_INFO: `/v3/projects/3ab69a8ea9d94395a6b9ae0110293214/users/cdcce87a6587441884e93573b6bd6d5e/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:44.291516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202167ee-f065-4821-8af3-f59d265b5bd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ab69a8ea9d94395a6b9ae0110293214, user_id=cdcce87a6587441884e93573b6bd6d5e, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:44.293365 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202167ee-f065-4821-8af3-f59d265b5bd7 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:44.305723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162eff0-2c03-4666-91c6-6fec177c5ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:44.309801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162eff0-2c03-4666-91c6-6fec177c5ada None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:44.309801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162eff0-2c03-4666-91c6-6fec177c5ada None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:44.309801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162eff0-2c03-4666-91c6-6fec177c5ada None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd/users/cdcce87a6587441884e93573b6bd6d5e/roles/d241ebe8f5654f46a5a92be6f6b0cc36` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:44.309801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162eff0-2c03-4666-91c6-6fec177c5ada None admin] RBAC: Authorizing `identity:create_grant(project_id=2710654d490f43b0adadfd7a62df84bd, user_id=cdcce87a6587441884e93573b6bd6d5e, role_id=d241ebe8f5654f46a5a92be6f6b0cc36)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:44.311425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162eff0-2c03-4666-91c6-6fec177c5ada None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:44.349464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 73/135] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:01:44 2022] PUT /identity/v3/projects/3ab69a8ea9d94395a6b9ae0110293214/users/cdcce87a6587441884e93573b6bd6d5e/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:01:44.350748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202167ee-f065-4821-8af3-f59d265b5bd7 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:44.370944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 63/136] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:01:43 2022] PUT /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd/users/cdcce87a6587441884e93573b6bd6d5e/roles/d241ebe8f5654f46a5a92be6f6b0cc36 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:01:44.503040 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202167ee-f065-4821-8af3-f59d265b5bd7 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:44.528631 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187df384-64a1-45aa-96ea-bcafb31e9da9 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:44.529013 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187df384-64a1-45aa-96ea-bcafb31e9da9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:44.529316 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187df384-64a1-45aa-96ea-bcafb31e9da9 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:44.531645 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 64/137] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:01:44.534638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fc142-4b32-41b0-8e8e-3a8d81206b10 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:44.534938 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fc142-4b32-41b0-8e8e-3a8d81206b10 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:44.535226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fc142-4b32-41b0-8e8e-3a8d81206b10 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:44.544743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 65/138] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:44 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:44.548659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba9c0d-277c-4af8-b366-769d57c1b3c2 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:44.548993 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba9c0d-277c-4af8-b366-769d57c1b3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:44.549269 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba9c0d-277c-4af8-b366-769d57c1b3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:44.567034 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555db3a-ef4a-45ab-8a8c-39f777d6a4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:44.568672 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555db3a-ef4a-45ab-8a8c-39f777d6a4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:44.568672 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555db3a-ef4a-45ab-8a8c-39f777d6a4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:44.568672 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555db3a-ef4a-45ab-8a8c-39f777d6a4a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:44.569219 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555db3a-ef4a-45ab-8a8c-39f777d6a4a3 None admin] RBAC: Adding query filter params (role.id=127c304ac7ee4fce8de777dbd4037b59, scope.project.id=2710654d490f43b0adadfd7a62df84bd, user.id=87dd1617d8f84685bd7cbcf2d4ac83ae) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:44.569540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555db3a-ef4a-45ab-8a8c-39f777d6a4a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:44.571130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555db3a-ef4a-45ab-8a8c-39f777d6a4a3 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:44.581641 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 74/139] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:01:44 2022] GET /identity/v3/role_assignments?user.id=87dd1617d8f84685bd7cbcf2d4ac83ae&scope.project.id=2710654d490f43b0adadfd7a62df84bd&role.id=127c304ac7ee4fce8de777dbd4037b59 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:44.581641 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ab7f-a028-4936-80a0-81663e7caab3 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:44.581641 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ab7f-a028-4936-80a0-81663e7caab3 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:44.581641 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ab7f-a028-4936-80a0-81663e7caab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:44.594096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-cdba9c0d-277c-4af8-b366-769d57c1b3c2 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:44.597168 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-cdba9c0d-277c-4af8-b366-769d57c1b3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:44.600027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-7519ab7f-a028-4936-80a0-81663e7caab3 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:44.601291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7519ab7f-a028-4936-80a0-81663e7caab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:44.645735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:44.687052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 75/140] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:44.713604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519ab7f-a028-4936-80a0-81663e7caab3 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:44.719562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 66/141] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:44.749997 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba9c0d-277c-4af8-b366-769d57c1b3c2 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:44.838187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519ab7f-a028-4936-80a0-81663e7caab3 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:44.889154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4e08a-980d-470f-97f6-54ebf79fe543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:44.889903 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4e08a-980d-470f-97f6-54ebf79fe543 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:44.890194 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4e08a-980d-470f-97f6-54ebf79fe543 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:44.890470 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4e08a-980d-470f-97f6-54ebf79fe543 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:44.916717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba9c0d-277c-4af8-b366-769d57c1b3c2 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:44.919900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4e08a-980d-470f-97f6-54ebf79fe543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:44.921271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-81b4e08a-980d-470f-97f6-54ebf79fe543 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:44.939703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:44.940772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:44.941592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:44.941592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:44.941592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:44.941592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:44.941592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:44.941592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:44.941592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:44.941592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:44.941592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:44.941592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:44.941592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:44.941592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:01:44.945606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 76/142] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:44.950896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4e08a-980d-470f-97f6-54ebf79fe543 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:44.987585 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c2170-97be-4723-abf1-82acd9ee545e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:44.988801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c2170-97be-4723-abf1-82acd9ee545e None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:44.989324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c2170-97be-4723-abf1-82acd9ee545e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:44.989820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c2170-97be-4723-abf1-82acd9ee545e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:44.992230 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c2170-97be-4723-abf1-82acd9ee545e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:44.993970 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c2170-97be-4723-abf1-82acd9ee545e None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:44.995887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 67/143] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:01:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:45.005633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c2170-97be-4723-abf1-82acd9ee545e None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:45.022149 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:45.193892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4e08a-980d-470f-97f6-54ebf79fe543 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:45.232804 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c2170-97be-4723-abf1-82acd9ee545e None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:45.293738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6e1a5-f7e8-40d3-971a-77f5d158c8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:45.294437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6e1a5-f7e8-40d3-971a-77f5d158c8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:45.294729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6e1a5-f7e8-40d3-971a-77f5d158c8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:45.295011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6e1a5-f7e8-40d3-971a-77f5d158c8b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:45.295787 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6e1a5-f7e8-40d3-971a-77f5d158c8b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:45.296097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6e1a5-f7e8-40d3-971a-77f5d158c8b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:45.297553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6e1a5-f7e8-40d3-971a-77f5d158c8b8 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:45.306684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f6f43-db6a-4cd8-ab1d-213d4e936c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:45.307558 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f6f43-db6a-4cd8-ab1d-213d4e936c2e None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:45.307850 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f6f43-db6a-4cd8-ab1d-213d4e936c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:45.308127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f6f43-db6a-4cd8-ab1d-213d4e936c2e None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:45.308866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f6f43-db6a-4cd8-ab1d-213d4e936c2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:45.310521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f6f43-db6a-4cd8-ab1d-213d4e936c2e None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:45.312277 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 77/144] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:01:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:45.324190 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6e1a5-f7e8-40d3-971a-77f5d158c8b8 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:45.398497 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 68/145] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:01:45 2022] POST /identity/v3/users => generated 283 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:01:45.487597 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:45.499314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6e1a5-f7e8-40d3-971a-77f5d158c8b8 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:45.559472 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7edf9ca-71da-4ac6-8630-e4da1bf2589d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:45.560377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edf9ca-71da-4ac6-8630-e4da1bf2589d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:45.560662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edf9ca-71da-4ac6-8630-e4da1bf2589d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:45.560940 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7edf9ca-71da-4ac6-8630-e4da1bf2589d None admin] PATH_INFO: `/v3/users/cdcce87a6587441884e93573b6bd6d5e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:45.564182 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edf9ca-71da-4ac6-8630-e4da1bf2589d None admin] RBAC: Authorizing `identity:get_user(user_id=cdcce87a6587441884e93573b6bd6d5e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:45.565771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7edf9ca-71da-4ac6-8630-e4da1bf2589d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:45.567687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 78/146] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:45 2022] GET /identity/v3/users/cdcce87a6587441884e93573b6bd6d5e => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:45.574271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f6f43-db6a-4cd8-ab1d-213d4e936c2e None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:45.629527 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:45.725508 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f6f43-db6a-4cd8-ab1d-213d4e936c2e None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:45.779828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9835c8-8d2c-4bc7-89cd-0b2c9d45a05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:45.780870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9835c8-8d2c-4bc7-89cd-0b2c9d45a05b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:45.781403 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9835c8-8d2c-4bc7-89cd-0b2c9d45a05b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:45.781887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9835c8-8d2c-4bc7-89cd-0b2c9d45a05b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:45.856624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2a611-2ebc-4713-b3d8-6518c134ff41 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:45.856917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2a611-2ebc-4713-b3d8-6518c134ff41 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:45.857239 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2a611-2ebc-4713-b3d8-6518c134ff41 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:45.858901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 79/147] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:45.867918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24d2be-c402-4956-bad4-41ae084dc870 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:45.868207 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24d2be-c402-4956-bad4-41ae084dc870 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:45.868491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24d2be-c402-4956-bad4-41ae084dc870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:45.872884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9835c8-8d2c-4bc7-89cd-0b2c9d45a05b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:45.874636 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9835c8-8d2c-4bc7-89cd-0b2c9d45a05b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:45.909992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:45.910177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-cf24d2be-c402-4956-bad4-41ae084dc870 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:45.911426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-cf24d2be-c402-4956-bad4-41ae084dc870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:45.996198 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 69/148] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:01:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:01:46.003450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9835c8-8d2c-4bc7-89cd-0b2c9d45a05b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:46.067702 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 80/149] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:01:46.160796 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9835c8-8d2c-4bc7-89cd-0b2c9d45a05b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:46.195749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24d2be-c402-4956-bad4-41ae084dc870 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:46.211657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:46.232132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5c5eb-cb32-4240-8d21-a28439778e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:46.233058 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5c5eb-cb32-4240-8d21-a28439778e0d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:46.233345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5c5eb-cb32-4240-8d21-a28439778e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:46.233619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5c5eb-cb32-4240-8d21-a28439778e0d None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:46.236019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5c5eb-cb32-4240-8d21-a28439778e0d None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:46.239722 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5c5eb-cb32-4240-8d21-a28439778e0d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:46.241709 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 70/150] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:46 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:46.249729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5c5eb-cb32-4240-8d21-a28439778e0d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:46.387279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:46.403727 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24d2be-c402-4956-bad4-41ae084dc870 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:46.406559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5c5eb-cb32-4240-8d21-a28439778e0d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:46.450271 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f923a-faaa-46ea-9f3d-4ab8db2ea1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:46.451081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f923a-faaa-46ea-9f3d-4ab8db2ea1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:46.451368 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f923a-faaa-46ea-9f3d-4ab8db2ea1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:46.451646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f923a-faaa-46ea-9f3d-4ab8db2ea1c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:46.464661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f923a-faaa-46ea-9f3d-4ab8db2ea1c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:46.466059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc857e-e640-4558-bf5a-8564391c3d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:46.466729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc857e-e640-4558-bf5a-8564391c3d7f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:46.467010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc857e-e640-4558-bf5a-8564391c3d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:46.467283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc857e-e640-4558-bf5a-8564391c3d7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:46.468133 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc857e-e640-4558-bf5a-8564391c3d7f None admin] RBAC: Adding query filter params (role.id=c91413ec3feb454ca7186eb52a57ed96, scope.project.id=2710654d490f43b0adadfd7a62df84bd, user.id=cdcce87a6587441884e93573b6bd6d5e) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:46.468446 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc857e-e640-4558-bf5a-8564391c3d7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:46.470188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc857e-e640-4558-bf5a-8564391c3d7f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-155f923a-faaa-46ea-9f3d-4ab8db2ea1c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:46.472950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:46.473811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:46.474567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:46.474567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:46.474567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:46.474567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:46.474567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:46.474567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:46.474567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:46.474567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:46.474567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:46.474567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:46.474567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:46.474567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:01:46.478455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 81/151] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:01:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:46.480582 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 71/152] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:01:46 2022] GET /identity/v3/role_assignments?user.id=cdcce87a6587441884e93573b6bd6d5e&scope.project.id=2710654d490f43b0adadfd7a62df84bd&role.id=c91413ec3feb454ca7186eb52a57ed96 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:46.487226 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f923a-faaa-46ea-9f3d-4ab8db2ea1c5 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:46.639105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f923a-faaa-46ea-9f3d-4ab8db2ea1c5 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:46.682015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e73501-cfb1-49c2-afb7-8669ccb25f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:46.682701 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e73501-cfb1-49c2-afb7-8669ccb25f76 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:46.682981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e73501-cfb1-49c2-afb7-8669ccb25f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:46.683256 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e73501-cfb1-49c2-afb7-8669ccb25f76 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:46.684016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e73501-cfb1-49c2-afb7-8669ccb25f76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:46.684324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e73501-cfb1-49c2-afb7-8669ccb25f76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:46.685748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e73501-cfb1-49c2-afb7-8669ccb25f76 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:46.697212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 82/153] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:46.702523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc857e-e640-4558-bf5a-8564391c3d7f None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:46.779802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:46.839514 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc857e-e640-4558-bf5a-8564391c3d7f None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:46.880659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5ee89-58bc-4d33-a482-bff43402e437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:46.881662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5ee89-58bc-4d33-a482-bff43402e437 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:46.881942 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5ee89-58bc-4d33-a482-bff43402e437 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:46.882217 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5ee89-58bc-4d33-a482-bff43402e437 None admin] PATH_INFO: `/v3/users/ede33ec052df4b3dab11128e172fd980` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:46.889846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5ee89-58bc-4d33-a482-bff43402e437 None admin] RBAC: Authorizing `identity:get_user(user_id=ede33ec052df4b3dab11128e172fd980)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:46.891245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5ee89-58bc-4d33-a482-bff43402e437 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:46.894138 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 72/154] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:46 2022] GET /identity/v3/users/ede33ec052df4b3dab11128e172fd980 => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:46.899705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e73501-cfb1-49c2-afb7-8669ccb25f76 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:46.922385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:47.031761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e73501-cfb1-49c2-afb7-8669ccb25f76 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:47.076638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd1d5f-1be6-479c-92cf-b6bab4016023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:47.077356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd1d5f-1be6-479c-92cf-b6bab4016023 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:47.077638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd1d5f-1be6-479c-92cf-b6bab4016023 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:47.077913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd1d5f-1be6-479c-92cf-b6bab4016023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:47.163827 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd1d5f-1be6-479c-92cf-b6bab4016023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:47.165329 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd1d5f-1be6-479c-92cf-b6bab4016023 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:47.185382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:47.284057 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 83/155] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:01:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:01:47.289712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5ee89-58bc-4d33-a482-bff43402e437 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:47.414555 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5ee89-58bc-4d33-a482-bff43402e437 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:47.445589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:47.455719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17fa4f-828b-48fa-a8d7-ad05aabdb03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:47.456780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17fa4f-828b-48fa-a8d7-ad05aabdb03a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:47.457089 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17fa4f-828b-48fa-a8d7-ad05aabdb03a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:47.457363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17fa4f-828b-48fa-a8d7-ad05aabdb03a None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:47.459491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa17fa4f-828b-48fa-a8d7-ad05aabdb03a None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:47.461171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa17fa4f-828b-48fa-a8d7-ad05aabdb03a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:47.463012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 73/156] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:47 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:47.469520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd1d5f-1be6-479c-92cf-b6bab4016023 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:47.553059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289aec03-65b1-4d1b-acca-6ec64bf07ce5 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:47.553385 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289aec03-65b1-4d1b-acca-6ec64bf07ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:47.553616 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289aec03-65b1-4d1b-acca-6ec64bf07ce5 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:47.555130 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 74/157] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:01:47.560846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2d680-f062-42b3-a9f2-d082cc8c6764 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:47.561176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2d680-f062-42b3-a9f2-d082cc8c6764 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:47.561426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2d680-f062-42b3-a9f2-d082cc8c6764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:47.591510 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-9ee2d680-f062-42b3-a9f2-d082cc8c6764 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:47.592785 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9ee2d680-f062-42b3-a9f2-d082cc8c6764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:47.632236 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd1d5f-1be6-479c-92cf-b6bab4016023 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:47.648048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:47.682025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1da950-ccb3-45dd-9837-a570f6380db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:47.682615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1da950-ccb3-45dd-9837-a570f6380db6 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:47.682894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1da950-ccb3-45dd-9837-a570f6380db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:47.683203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1da950-ccb3-45dd-9837-a570f6380db6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:47.684044 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1da950-ccb3-45dd-9837-a570f6380db6 None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=2710654d490f43b0adadfd7a62df84bd, user.id=ede33ec052df4b3dab11128e172fd980) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:47.684362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1da950-ccb3-45dd-9837-a570f6380db6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:47.685857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1da950-ccb3-45dd-9837-a570f6380db6 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:47.701441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 84/158] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:01:47 2022] GET /identity/v3/role_assignments?user.id=ede33ec052df4b3dab11128e172fd980&scope.project.id=2710654d490f43b0adadfd7a62df84bd&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:47.743570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 75/159] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:01:47.783759 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1da950-ccb3-45dd-9837-a570f6380db6 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:47.909548 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1da950-ccb3-45dd-9837-a570f6380db6 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:47.952650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75d956-f0b0-48a7-b195-21d26b37d2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:47.953428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75d956-f0b0-48a7-b195-21d26b37d2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:47.953711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75d956-f0b0-48a7-b195-21d26b37d2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:47.953990 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75d956-f0b0-48a7-b195-21d26b37d2b0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:47.977301 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a75d956-f0b0-48a7-b195-21d26b37d2b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-6a75d956-f0b0-48a7-b195-21d26b37d2b0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:47.986223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:47.987165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:47.988003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:47.988003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:47.988003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:47.988003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:47.988003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:47.988003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:47.988003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:47.988003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:47.988003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:47.988003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:47.988003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:01:47.988003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:01:47.988003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 85/160] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:47.994044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2d680-f062-42b3-a9f2-d082cc8c6764 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:48.131017 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2d680-f062-42b3-a9f2-d082cc8c6764 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:48.153141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d7a75-835b-4ce7-8b85-e5ab2a172405 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:48.153813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d7a75-835b-4ce7-8b85-e5ab2a172405 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:48.154286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d7a75-835b-4ce7-8b85-e5ab2a172405 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:48.156016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 86/161] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:01:48.158332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:48.165259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af98151-3cf8-4a73-bfd0-cb7293ffe8f5 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:48.165811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af98151-3cf8-4a73-bfd0-cb7293ffe8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:48.171126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af98151-3cf8-4a73-bfd0-cb7293ffe8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:48.172501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b2a7b-3299-45d4-9923-97c1b4f85376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:48.173248 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b2a7b-3299-45d4-9923-97c1b4f85376 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:48.173627 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b2a7b-3299-45d4-9923-97c1b4f85376 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:48.173931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b2a7b-3299-45d4-9923-97c1b4f85376 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:48.175032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b2a7b-3299-45d4-9923-97c1b4f85376 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:48.175145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b2a7b-3299-45d4-9923-97c1b4f85376 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:48.177824 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b2a7b-3299-45d4-9923-97c1b4f85376 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:48.185821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 76/162] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:01:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:48.187532 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-5af98151-3cf8-4a73-bfd0-cb7293ffe8f5 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:48.189703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5af98151-3cf8-4a73-bfd0-cb7293ffe8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:48.192483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b2a7b-3299-45d4-9923-97c1b4f85376 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:48.229529 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:48.274198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 87/163] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:48.301091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af98151-3cf8-4a73-bfd0-cb7293ffe8f5 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:48.427495 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b2a7b-3299-45d4-9923-97c1b4f85376 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:48.430157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af98151-3cf8-4a73-bfd0-cb7293ffe8f5 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:48.477779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0fd4c-15ee-4e60-b70f-908c2e41f8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:48.479598 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0fd4c-15ee-4e60-b70f-908c2e41f8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:48.480159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0fd4c-15ee-4e60-b70f-908c2e41f8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:48.480631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0fd4c-15ee-4e60-b70f-908c2e41f8d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:48.493824 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0fd4c-15ee-4e60-b70f-908c2e41f8d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-b2d0fd4c-15ee-4e60-b70f-908c2e41f8d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:48.503661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:48.504542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:48.505416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:48.505416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:48.505416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:48.505416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:48.505416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:48.505416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:48.505416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:48.505416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:48.505416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:48.505416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:48.505416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:48.505416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:01:48.508280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 88/164] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:48.514055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174b550-0cf3-4b8a-a394-edd075edd546 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:48.514610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174b550-0cf3-4b8a-a394-edd075edd546 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:48.515109 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174b550-0cf3-4b8a-a394-edd075edd546 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:48.517354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 89/165] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:48.519269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174b550-0cf3-4b8a-a394-edd075edd546 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:48.546358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20dea5-cce3-4180-842a-647ba1c42f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:48.547261 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20dea5-cce3-4180-842a-647ba1c42f53 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:48.547543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20dea5-cce3-4180-842a-647ba1c42f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:48.548224 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20dea5-cce3-4180-842a-647ba1c42f53 None admin] PATH_INFO: `/v3/users/87dd1617d8f84685bd7cbcf2d4ac83ae` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:48.549707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20dea5-cce3-4180-842a-647ba1c42f53 None admin] RBAC: Authorizing `identity:get_user(user_id=87dd1617d8f84685bd7cbcf2d4ac83ae)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:48.552756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d20dea5-cce3-4180-842a-647ba1c42f53 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:48.556125 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 77/166] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:48 2022] GET /identity/v3/users/87dd1617d8f84685bd7cbcf2d4ac83ae => generated 289 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:48.558396 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aca4a3-c995-451a-b639-f3122a35813e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:48.559742 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aca4a3-c995-451a-b639-f3122a35813e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:48.562179 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:48.563503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aca4a3-c995-451a-b639-f3122a35813e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:48.594275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-95aca4a3-c995-451a-b639-f3122a35813e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:48.594275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-95aca4a3-c995-451a-b639-f3122a35813e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:48.691918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174b550-0cf3-4b8a-a394-edd075edd546 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:48.703593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:48.733659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692e5e9-9ff1-4f8d-84cd-3c5f3344362c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:48.734658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692e5e9-9ff1-4f8d-84cd-3c5f3344362c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:48.736189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692e5e9-9ff1-4f8d-84cd-3c5f3344362c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:48.736671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692e5e9-9ff1-4f8d-84cd-3c5f3344362c None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:48.737928 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692e5e9-9ff1-4f8d-84cd-3c5f3344362c None admin] RBAC: Adding query filter params (name=member) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:48.739277 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692e5e9-9ff1-4f8d-84cd-3c5f3344362c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:48.742247 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692e5e9-9ff1-4f8d-84cd-3c5f3344362c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:48.759227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 78/167] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:48.760426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aca4a3-c995-451a-b639-f3122a35813e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:48.764585 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 90/168] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:01:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:48.775922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692e5e9-9ff1-4f8d-84cd-3c5f3344362c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:48.908922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:48.919179 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aca4a3-c995-451a-b639-f3122a35813e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:48.921587 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692e5e9-9ff1-4f8d-84cd-3c5f3344362c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:48.956373 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974b5fd-6c61-48ce-acb7-30e2dbb12fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:48.957672 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974b5fd-6c61-48ce-acb7-30e2dbb12fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:48.958168 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974b5fd-6c61-48ce-acb7-30e2dbb12fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:48.958638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974b5fd-6c61-48ce-acb7-30e2dbb12fc1 None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:48.960152 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974b5fd-6c61-48ce-acb7-30e2dbb12fc1 None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:48.961888 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974b5fd-6c61-48ce-acb7-30e2dbb12fc1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:48.963865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 79/169] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:48 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:48.964837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974b5fd-6c61-48ce-acb7-30e2dbb12fc1 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:48.966968 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dbafb-16fc-47ec-b385-f597fb74c133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:48.967973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dbafb-16fc-47ec-b385-f597fb74c133 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:48.968261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dbafb-16fc-47ec-b385-f597fb74c133 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:48.968534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dbafb-16fc-47ec-b385-f597fb74c133 None admin] PATH_INFO: `/v3/users/cdcce87a6587441884e93573b6bd6d5e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:48.970498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407dbafb-16fc-47ec-b385-f597fb74c133 None admin] RBAC: Authorizing `identity:get_user(user_id=cdcce87a6587441884e93573b6bd6d5e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:48.972166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407dbafb-16fc-47ec-b385-f597fb74c133 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:48.977011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 91/170] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:48 2022] GET /identity/v3/users/cdcce87a6587441884e93573b6bd6d5e => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:48.981943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dbafb-16fc-47ec-b385-f597fb74c133 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:49.028553 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:49.081952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974b5fd-6c61-48ce-acb7-30e2dbb12fc1 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:49.120992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afc019-0b36-446a-b16b-43b57aad49a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:49.122178 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afc019-0b36-446a-b16b-43b57aad49a0 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:49.122668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afc019-0b36-446a-b16b-43b57aad49a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:49.123245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afc019-0b36-446a-b16b-43b57aad49a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:49.134837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66afc019-0b36-446a-b16b-43b57aad49a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-66afc019-0b36-446a-b16b-43b57aad49a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:49.144444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:49.145399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:49.146193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:49.146193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:49.146193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:49.146193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:49.146193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:49.146193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:49.146193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:49.146193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:49.146193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:49.146193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:49.146193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:01:49.146193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:01:49.148622 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 80/171] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:01:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:49.149806 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afc019-0b36-446a-b16b-43b57aad49a0 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:49.207236 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dbafb-16fc-47ec-b385-f597fb74c133 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:49.263583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afc019-0b36-446a-b16b-43b57aad49a0 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:49.278516 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:49.302853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee8b32-a628-450e-ad8b-e93a46a18efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:49.303857 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee8b32-a628-450e-ad8b-e93a46a18efa None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:49.304369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee8b32-a628-450e-ad8b-e93a46a18efa None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:49.304903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee8b32-a628-450e-ad8b-e93a46a18efa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:49.306247 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b0218-ece1-4bc9-abbb-885823a0bc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:49.307157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b0218-ece1-4bc9-abbb-885823a0bc0d None admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:49.307439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b0218-ece1-4bc9-abbb-885823a0bc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:49.307719 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b0218-ece1-4bc9-abbb-885823a0bc0d None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd/users/87dd1617d8f84685bd7cbcf2d4ac83ae/roles/127c304ac7ee4fce8de777dbd4037b59` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:49.314891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b0218-ece1-4bc9-abbb-885823a0bc0d None admin] RBAC: Authorizing `identity:create_grant(project_id=2710654d490f43b0adadfd7a62df84bd, user_id=87dd1617d8f84685bd7cbcf2d4ac83ae, role_id=127c304ac7ee4fce8de777dbd4037b59)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:49.316823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b0218-ece1-4bc9-abbb-885823a0bc0d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:49.382882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee8b32-a628-450e-ad8b-e93a46a18efa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:49.384882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee8b32-a628-450e-ad8b-e93a46a18efa None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:49.411548 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 92/172] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:01:48 2022] PUT /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd/users/87dd1617d8f84685bd7cbcf2d4ac83ae/roles/127c304ac7ee4fce8de777dbd4037b59 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:01:49.412458 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b0218-ece1-4bc9-abbb-885823a0bc0d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:49.474748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:49.588717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 81/173] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:01:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:01:49.594196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee8b32-a628-450e-ad8b-e93a46a18efa None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:49.621314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:49.686140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b0218-ece1-4bc9-abbb-885823a0bc0d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:49.754727 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7acf5-6884-426e-a1e9-c68311fd3dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:49.755494 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7acf5-6884-426e-a1e9-c68311fd3dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:49.755685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7acf5-6884-426e-a1e9-c68311fd3dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:49.755963 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7acf5-6884-426e-a1e9-c68311fd3dd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:49.756731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7acf5-6884-426e-a1e9-c68311fd3dd0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:49.757314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7acf5-6884-426e-a1e9-c68311fd3dd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:49.758737 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7acf5-6884-426e-a1e9-c68311fd3dd0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:49.775398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 93/174] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:01:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:49.777452 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee8b32-a628-450e-ad8b-e93a46a18efa None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:49.781069 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7acf5-6884-426e-a1e9-c68311fd3dd0 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:49.861302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa26f9-2f94-4ffa-b08a-4880c3a6bd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:49.864984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa26f9-2f94-4ffa-b08a-4880c3a6bd2a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:49.864984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa26f9-2f94-4ffa-b08a-4880c3a6bd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:49.864984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa26f9-2f94-4ffa-b08a-4880c3a6bd2a None admin] PATH_INFO: `/v3/projects/3ab69a8ea9d94395a6b9ae0110293214` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:49.868071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa26f9-2f94-4ffa-b08a-4880c3a6bd2a None admin] RBAC: Authorizing `identity:get_project(project_id=3ab69a8ea9d94395a6b9ae0110293214)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:49.873409 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa26f9-2f94-4ffa-b08a-4880c3a6bd2a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:49.875276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 82/175] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:49 2022] GET /identity/v3/projects/3ab69a8ea9d94395a6b9ae0110293214 => generated 312 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:49.886910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa26f9-2f94-4ffa-b08a-4880c3a6bd2a None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:50.001935 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:50.013340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:50.023317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa26f9-2f94-4ffa-b08a-4880c3a6bd2a None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:50.043958 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7acf5-6884-426e-a1e9-c68311fd3dd0 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:50.061828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797a3e8-9562-48ab-b8ed-6c63e6d53b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:50.062505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797a3e8-9562-48ab-b8ed-6c63e6d53b8f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:50.062805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797a3e8-9562-48ab-b8ed-6c63e6d53b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:50.063067 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797a3e8-9562-48ab-b8ed-6c63e6d53b8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:50.064534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797a3e8-9562-48ab-b8ed-6c63e6d53b8f None admin] RBAC: Adding query filter params (role.id=c91413ec3feb454ca7186eb52a57ed96, scope.project.id=3ab69a8ea9d94395a6b9ae0110293214, user.id=cdcce87a6587441884e93573b6bd6d5e) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:50.067404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797a3e8-9562-48ab-b8ed-6c63e6d53b8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:50.067404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797a3e8-9562-48ab-b8ed-6c63e6d53b8f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:50.075158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 83/176] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:01:49 2022] GET /identity/v3/role_assignments?user.id=cdcce87a6587441884e93573b6bd6d5e&scope.project.id=3ab69a8ea9d94395a6b9ae0110293214&role.id=c91413ec3feb454ca7186eb52a57ed96 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:50.132529 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28af974-681d-4495-9a2a-5490e9a78de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:50.133352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28af974-681d-4495-9a2a-5490e9a78de0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:50.133637 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28af974-681d-4495-9a2a-5490e9a78de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:50.133916 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28af974-681d-4495-9a2a-5490e9a78de0 None admin] PATH_INFO: `/v3/users/cdcce87a6587441884e93573b6bd6d5e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:50.137797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28af974-681d-4495-9a2a-5490e9a78de0 None admin] RBAC: Authorizing `identity:get_user(user_id=cdcce87a6587441884e93573b6bd6d5e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:50.139297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28af974-681d-4495-9a2a-5490e9a78de0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:50.141104 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 94/177] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:49 2022] GET /identity/v3/users/cdcce87a6587441884e93573b6bd6d5e => generated 275 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:50.146975 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797a3e8-9562-48ab-b8ed-6c63e6d53b8f None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:50.186657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03feaa23-25d9-4cc8-b744-6d9275b31949 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:50.186974 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03feaa23-25d9-4cc8-b744-6d9275b31949 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:50.187242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03feaa23-25d9-4cc8-b744-6d9275b31949 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:50.189593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 95/178] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:01:50.195278 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489e576-34c6-4e68-a69c-6932480ce422 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:50.195609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489e576-34c6-4e68-a69c-6932480ce422 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:50.195837 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489e576-34c6-4e68-a69c-6932480ce422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:50.312432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-2489e576-34c6-4e68-a69c-6932480ce422 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:50.314683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2489e576-34c6-4e68-a69c-6932480ce422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:50.385054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797a3e8-9562-48ab-b8ed-6c63e6d53b8f None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:50.406151 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 96/179] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:01:50.408741 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ac7c6-ff3a-40df-a16f-2fe24cbc3caf None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:50.411296 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ac7c6-ff3a-40df-a16f-2fe24cbc3caf None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:50.411296 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ac7c6-ff3a-40df-a16f-2fe24cbc3caf None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:50.411296 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 97/180] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:50.412399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a84ff1-26ca-4275-8579-b0b106f05bdf None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:50.412900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a84ff1-26ca-4275-8579-b0b106f05bdf None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:50.413456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a84ff1-26ca-4275-8579-b0b106f05bdf None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:50.415970 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 98/181] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:50.421566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85145316-2f69-4243-97c3-31f8cd366870 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:50.421858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85145316-2f69-4243-97c3-31f8cd366870 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:50.422145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85145316-2f69-4243-97c3-31f8cd366870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:50.428794 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efdb43-eab4-4a02-8d6d-e352bf73ebc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:50.429501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efdb43-eab4-4a02-8d6d-e352bf73ebc7 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:50.429790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efdb43-eab4-4a02-8d6d-e352bf73ebc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:50.430069 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1efdb43-eab4-4a02-8d6d-e352bf73ebc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:50.440660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-85145316-2f69-4243-97c3-31f8cd366870 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:50.442009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-85145316-2f69-4243-97c3-31f8cd366870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:50.448008 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:50.515199 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efdb43-eab4-4a02-8d6d-e352bf73ebc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:50.517380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1efdb43-eab4-4a02-8d6d-e352bf73ebc7 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:50.540991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 99/182] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:50.540991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da0c1c-43fe-484f-9934-2bac5b790c0e None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:50.540991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da0c1c-43fe-484f-9934-2bac5b790c0e None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:50.540991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da0c1c-43fe-484f-9934-2bac5b790c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:50.557813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-20da0c1c-43fe-484f-9934-2bac5b790c0e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:50.559107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-20da0c1c-43fe-484f-9934-2bac5b790c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:50.561647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:50.633569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 100/183] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:50.641251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da0c1c-43fe-484f-9934-2bac5b790c0e None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:50.641348 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 84/184] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:01:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:01:50.642253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efdb43-eab4-4a02-8d6d-e352bf73ebc7 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:50.694251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:50.746101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da0c1c-43fe-484f-9934-2bac5b790c0e None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:50.755508 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1efdb43-eab4-4a02-8d6d-e352bf73ebc7 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:50.780239 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108bed9-8f61-4614-bae0-3317c9c3f21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:50.781070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108bed9-8f61-4614-bae0-3317c9c3f21e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:50.781364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108bed9-8f61-4614-bae0-3317c9c3f21e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:50.781644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108bed9-8f61-4614-bae0-3317c9c3f21e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:50.788862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:50.792749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108bed9-8f61-4614-bae0-3317c9c3f21e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:50.793657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50d4fa-db57-4d7c-b93a-cb94aa89e1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:50.799691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50d4fa-db57-4d7c-b93a-cb94aa89e1ca None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:50.799691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50d4fa-db57-4d7c-b93a-cb94aa89e1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:50.799691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50d4fa-db57-4d7c-b93a-cb94aa89e1ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-8108bed9-8f61-4614-bae0-3317c9c3f21e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:50.801527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:50.802399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:50.803650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:50.803650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:50.803650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:50.803650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:50.803650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:50.803650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:50.803650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:50.803650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:50.803650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:50.803650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:50.803650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:50.803650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:01:50.803650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 101/185] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:01:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:50.803650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108bed9-8f61-4614-bae0-3317c9c3f21e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:50.812863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50d4fa-db57-4d7c-b93a-cb94aa89e1ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-9d50d4fa-db57-4d7c-b93a-cb94aa89e1ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:50.821947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:50.822869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:50.823640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:50.823640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:50.823640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:50.823640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:50.823640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:50.823640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:50.823640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:50.823640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:50.823640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:50.823640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:50.823640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:01:50.823640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:01:50.823640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 85/186] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:01:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:50.824952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50d4fa-db57-4d7c-b93a-cb94aa89e1ca None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:50.922429 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108bed9-8f61-4614-bae0-3317c9c3f21e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:50.941055 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50d4fa-db57-4d7c-b93a-cb94aa89e1ca None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:50.944619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:50.962245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e31e13-0c45-4fc3-b79a-54fa9925da2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:50.963159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e31e13-0c45-4fc3-b79a-54fa9925da2d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:50.963444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e31e13-0c45-4fc3-b79a-54fa9925da2d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:50.963724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e31e13-0c45-4fc3-b79a-54fa9925da2d None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:50.964952 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e31e13-0c45-4fc3-b79a-54fa9925da2d None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:50.966460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e31e13-0c45-4fc3-b79a-54fa9925da2d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:50.968210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 102/187] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:50 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:50.969023 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e31e13-0c45-4fc3-b79a-54fa9925da2d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:50.985013 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06f4ba-edd4-44b6-b75a-a9c00e5960f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:50.986238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06f4ba-edd4-44b6-b75a-a9c00e5960f8 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:50.986778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06f4ba-edd4-44b6-b75a-a9c00e5960f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:50.987247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06f4ba-edd4-44b6-b75a-a9c00e5960f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:50.988670 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06f4ba-edd4-44b6-b75a-a9c00e5960f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:50.990447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06f4ba-edd4-44b6-b75a-a9c00e5960f8 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:50.992305 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 86/188] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:01:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:50.997595 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06f4ba-edd4-44b6-b75a-a9c00e5960f8 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:51.038558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:51.080986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e31e13-0c45-4fc3-b79a-54fa9925da2d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:51.116028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06f4ba-edd4-44b6-b75a-a9c00e5960f8 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:51.121444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc5dd0-5969-4d84-84fa-59e5de197dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:51.122373 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc5dd0-5969-4d84-84fa-59e5de197dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:51.122841 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc5dd0-5969-4d84-84fa-59e5de197dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:51.123357 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc5dd0-5969-4d84-84fa-59e5de197dd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:51.124289 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc5dd0-5969-4d84-84fa-59e5de197dd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:51.124783 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc5dd0-5969-4d84-84fa-59e5de197dd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:51.127450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc5dd0-5969-4d84-84fa-59e5de197dd5 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:51.136645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 103/189] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:51.137611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc5dd0-5969-4d84-84fa-59e5de197dd5 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:51.155489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d667a-e560-4e01-9489-44f55e36b0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:51.156300 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d667a-e560-4e01-9489-44f55e36b0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:51.156581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d667a-e560-4e01-9489-44f55e36b0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:51.156884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d667a-e560-4e01-9489-44f55e36b0c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:51.158056 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d667a-e560-4e01-9489-44f55e36b0c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:51.158367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d667a-e560-4e01-9489-44f55e36b0c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:51.159764 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d667a-e560-4e01-9489-44f55e36b0c5 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:51.168782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 87/190] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:01:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:51.174908 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d667a-e560-4e01-9489-44f55e36b0c5 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:51.252513 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:51.293593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc5dd0-5969-4d84-84fa-59e5de197dd5 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:51.321342 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:51.321737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d667a-e560-4e01-9489-44f55e36b0c5 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:51.333631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2152be-1186-4ccd-90e6-25711f194333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:51.334338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2152be-1186-4ccd-90e6-25711f194333 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:51.334629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2152be-1186-4ccd-90e6-25711f194333 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:51.334966 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2152be-1186-4ccd-90e6-25711f194333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:51.335829 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2152be-1186-4ccd-90e6-25711f194333 None admin] RBAC: Adding query filter params (role.id=d241ebe8f5654f46a5a92be6f6b0cc36, scope.project.id=2710654d490f43b0adadfd7a62df84bd, user.id=cdcce87a6587441884e93573b6bd6d5e) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:51.336148 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2152be-1186-4ccd-90e6-25711f194333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:51.337655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2152be-1186-4ccd-90e6-25711f194333 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:51.344123 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 104/191] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:01:51 2022] GET /identity/v3/role_assignments?user.id=cdcce87a6587441884e93573b6bd6d5e&scope.project.id=2710654d490f43b0adadfd7a62df84bd&role.id=d241ebe8f5654f46a5a92be6f6b0cc36 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:51.345569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2152be-1186-4ccd-90e6-25711f194333 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:51.365861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c56320-cb10-4b9b-9386-c09907f56cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:51.366692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c56320-cb10-4b9b-9386-c09907f56cac None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:51.366984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c56320-cb10-4b9b-9386-c09907f56cac None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:51.367267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c56320-cb10-4b9b-9386-c09907f56cac None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:51.368147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c56320-cb10-4b9b-9386-c09907f56cac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:51.369738 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c56320-cb10-4b9b-9386-c09907f56cac None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:51.470690 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2152be-1186-4ccd-90e6-25711f194333 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:51.481052 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 88/192] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:01:51 2022] POST /identity/v3/users => generated 297 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:01:51.482566 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c56320-cb10-4b9b-9386-c09907f56cac None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:51.534888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c8946-bf35-45e5-81cb-3234b00e2d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:51.536091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c8946-bf35-45e5-81cb-3234b00e2d6b None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:51.536523 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c8946-bf35-45e5-81cb-3234b00e2d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:51.536939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c8946-bf35-45e5-81cb-3234b00e2d6b None admin] PATH_INFO: `/v3/users/c1642bebaf594611827900f344ef5b74` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:51.554369 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c8946-bf35-45e5-81cb-3234b00e2d6b None admin] RBAC: Authorizing `identity:get_user(user_id=c1642bebaf594611827900f344ef5b74)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:51.555778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c8946-bf35-45e5-81cb-3234b00e2d6b None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:51.557658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 105/193] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:51 2022] GET /identity/v3/users/c1642bebaf594611827900f344ef5b74 => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:51.564226 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c8946-bf35-45e5-81cb-3234b00e2d6b None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:51.639049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:51.653614 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:51.666548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c56320-cb10-4b9b-9386-c09907f56cac None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:51.725877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c8946-bf35-45e5-81cb-3234b00e2d6b None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:51.730803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83c905-58cc-4f6d-ada4-c86fc7fab145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:51.731895 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83c905-58cc-4f6d-ada4-c86fc7fab145 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:51.732314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83c905-58cc-4f6d-ada4-c86fc7fab145 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:51.732723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83c905-58cc-4f6d-ada4-c86fc7fab145 None admin] PATH_INFO: `/v3/users/c1642bebaf594611827900f344ef5b74` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:51.734177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83c905-58cc-4f6d-ada4-c86fc7fab145 None admin] RBAC: Authorizing `identity:get_user(user_id=c1642bebaf594611827900f344ef5b74)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:51.736903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83c905-58cc-4f6d-ada4-c86fc7fab145 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:51.738835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 89/194] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:51 2022] GET /identity/v3/users/c1642bebaf594611827900f344ef5b74 => generated 283 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:51.744671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83c905-58cc-4f6d-ada4-c86fc7fab145 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:51.768002 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54acf2c6-5972-4fed-a575-398840f29c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:51.768675 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54acf2c6-5972-4fed-a575-398840f29c33 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:51.769128 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54acf2c6-5972-4fed-a575-398840f29c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:51.769408 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54acf2c6-5972-4fed-a575-398840f29c33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:51.877174 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54acf2c6-5972-4fed-a575-398840f29c33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:51.879354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54acf2c6-5972-4fed-a575-398840f29c33 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:51.887775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83c905-58cc-4f6d-ada4-c86fc7fab145 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:51.934340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bab556-37fd-4437-87d0-1bdc08ac7ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:51.935176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bab556-37fd-4437-87d0-1bdc08ac7ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:51.935564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bab556-37fd-4437-87d0-1bdc08ac7ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:51.936252 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bab556-37fd-4437-87d0-1bdc08ac7ac2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:51.959183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:51.980658 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:52.032837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bab556-37fd-4437-87d0-1bdc08ac7ac2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:52.034734 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bab556-37fd-4437-87d0-1bdc08ac7ac2 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:52.037796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 106/195] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:01:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 10:01:52.048523 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54acf2c6-5972-4fed-a575-398840f29c33 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:52.181459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 90/196] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:01:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 10:01:52.183788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecd0cd-06ba-4b7c-a555-8a2639d1ec66 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:52.184086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecd0cd-06ba-4b7c-a555-8a2639d1ec66 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:52.184368 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecd0cd-06ba-4b7c-a555-8a2639d1ec66 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:52.185503 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54acf2c6-5972-4fed-a575-398840f29c33 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:52.185716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 91/197] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:52.191376 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ecd0cd-06ba-4b7c-a555-8a2639d1ec66 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:52.226376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b291b-738f-4dd6-91b7-0c4af9685b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:52.227256 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b291b-738f-4dd6-91b7-0c4af9685b66 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:52.227539 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b291b-738f-4dd6-91b7-0c4af9685b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:52.227818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b291b-738f-4dd6-91b7-0c4af9685b66 None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:52.229287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b291b-738f-4dd6-91b7-0c4af9685b66 None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:52.230815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599b291b-738f-4dd6-91b7-0c4af9685b66 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:52.232657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 107/198] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:52 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:52.234045 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7f1f1-1dcc-4cff-840a-1893b23b8d05 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:52.234449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7f1f1-1dcc-4cff-840a-1893b23b8d05 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:52.234732 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7f1f1-1dcc-4cff-840a-1893b23b8d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:52.254750 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-95d7f1f1-1dcc-4cff-840a-1893b23b8d05 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:52.255968 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-95d7f1f1-1dcc-4cff-840a-1893b23b8d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:52.265389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:52.272534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:52.330953 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ecd0cd-06ba-4b7c-a555-8a2639d1ec66 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:52.338552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 108/199] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:52.339332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7f1f1-1dcc-4cff-840a-1893b23b8d05 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:52.379659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f387aab-1efa-42db-b57f-93324bdfe24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:52.380570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f387aab-1efa-42db-b57f-93324bdfe24b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:52.380865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f387aab-1efa-42db-b57f-93324bdfe24b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:52.381172 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f387aab-1efa-42db-b57f-93324bdfe24b None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:52.382441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f387aab-1efa-42db-b57f-93324bdfe24b None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:52.383894 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f387aab-1efa-42db-b57f-93324bdfe24b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:52.385816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 92/200] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:52 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:52.386646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f387aab-1efa-42db-b57f-93324bdfe24b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:52.468800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7f1f1-1dcc-4cff-840a-1893b23b8d05 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:52.508164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f387aab-1efa-42db-b57f-93324bdfe24b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:52.520158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e8f69-2637-4736-b8c0-eda9362114b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:52.521862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e8f69-2637-4736-b8c0-eda9362114b5 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:52.522357 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e8f69-2637-4736-b8c0-eda9362114b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:52.522958 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e8f69-2637-4736-b8c0-eda9362114b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:52.524026 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e8f69-2637-4736-b8c0-eda9362114b5 None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=354b4a7dec454bbe9294b154303e7f7e, user.id=c1642bebaf594611827900f344ef5b74) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:52.524553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e8f69-2637-4736-b8c0-eda9362114b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:52.526382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e8f69-2637-4736-b8c0-eda9362114b5 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:52.535370 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 109/201] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:01:52 2022] GET /identity/v3/role_assignments?user.id=c1642bebaf594611827900f344ef5b74&scope.project.id=354b4a7dec454bbe9294b154303e7f7e&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:52.537183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e8f69-2637-4736-b8c0-eda9362114b5 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:52.548849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:52.559651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045d943-8727-48b8-8a8d-175d39480c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:52.560492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045d943-8727-48b8-8a8d-175d39480c22 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:52.560928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045d943-8727-48b8-8a8d-175d39480c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:52.561427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045d943-8727-48b8-8a8d-175d39480c22 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:52.564264 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:52.575333 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045d943-8727-48b8-8a8d-175d39480c22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-e045d943-8727-48b8-8a8d-175d39480c22 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:52.584736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:52.585723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:52.586881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:52.586881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:52.586881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:52.586881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:52.586881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:52.586881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:52.586881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:52.586881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:52.586881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:52.586881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:52.586881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:01:52.586881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:01:52.586881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 93/202] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:52.599213 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045d943-8727-48b8-8a8d-175d39480c22 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:52.727540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045d943-8727-48b8-8a8d-175d39480c22 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:52.774646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbaf492-86d4-46ca-b600-7d71eb58d772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:52.775253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbaf492-86d4-46ca-b600-7d71eb58d772 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:52.775538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbaf492-86d4-46ca-b600-7d71eb58d772 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:52.775813 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbaf492-86d4-46ca-b600-7d71eb58d772 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:52.776593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbaf492-86d4-46ca-b600-7d71eb58d772 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:52.776904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbaf492-86d4-46ca-b600-7d71eb58d772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:52.778355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbaf492-86d4-46ca-b600-7d71eb58d772 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:52.790453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e8f69-2637-4736-b8c0-eda9362114b5 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:52.804821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 94/203] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:01:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:52.811462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbaf492-86d4-46ca-b600-7d71eb58d772 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:52.853710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7eb0e-39ef-4a57-a7e6-ca0f972ac2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:52.854779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7eb0e-39ef-4a57-a7e6-ca0f972ac2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:52.855268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7eb0e-39ef-4a57-a7e6-ca0f972ac2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:52.855751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7eb0e-39ef-4a57-a7e6-ca0f972ac2b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:52.856800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7eb0e-39ef-4a57-a7e6-ca0f972ac2b8 None admin] RBAC: Adding query filter params (role.id=d241ebe8f5654f46a5a92be6f6b0cc36, scope.project.id=354b4a7dec454bbe9294b154303e7f7e, user.id=c1642bebaf594611827900f344ef5b74) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:52.857355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7eb0e-39ef-4a57-a7e6-ca0f972ac2b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:52.859018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a7eb0e-39ef-4a57-a7e6-ca0f972ac2b8 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:52.866098 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 110/204] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:01:52 2022] GET /identity/v3/role_assignments?user.id=c1642bebaf594611827900f344ef5b74&scope.project.id=354b4a7dec454bbe9294b154303e7f7e&role.id=d241ebe8f5654f46a5a92be6f6b0cc36 => generated 259 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:52.870544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:52.931557 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbaf492-86d4-46ca-b600-7d71eb58d772 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:52.982633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5219c9-52f0-4a86-a85b-74fa26ecdbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:52.983651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5219c9-52f0-4a86-a85b-74fa26ecdbce None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:52.984074 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5219c9-52f0-4a86-a85b-74fa26ecdbce None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:52.984476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5219c9-52f0-4a86-a85b-74fa26ecdbce None admin] PATH_INFO: `/v3/users/87dd1617d8f84685bd7cbcf2d4ac83ae` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:52.989973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5219c9-52f0-4a86-a85b-74fa26ecdbce None admin] RBAC: Authorizing `identity:get_user(user_id=87dd1617d8f84685bd7cbcf2d4ac83ae)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:52.991531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5219c9-52f0-4a86-a85b-74fa26ecdbce None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:52.993669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 95/205] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:52 2022] GET /identity/v3/users/87dd1617d8f84685bd7cbcf2d4ac83ae => generated 289 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:52.999654 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7eb0e-39ef-4a57-a7e6-ca0f972ac2b8 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:53.112934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:53.140102 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7eb0e-39ef-4a57-a7e6-ca0f972ac2b8 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:53.191646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9794b-1faf-46b7-88e6-55cf1c63e1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:53.192699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9794b-1faf-46b7-88e6-55cf1c63e1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:53.193216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9794b-1faf-46b7-88e6-55cf1c63e1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:53.193702 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9794b-1faf-46b7-88e6-55cf1c63e1e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:53.279738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9794b-1faf-46b7-88e6-55cf1c63e1e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:53.281628 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9794b-1faf-46b7-88e6-55cf1c63e1e8 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:53.369066 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:53.394109 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 111/206] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:01:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:01:53.400161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5219c9-52f0-4a86-a85b-74fa26ecdbce None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:53.554871 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5219c9-52f0-4a86-a85b-74fa26ecdbce None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:53.572132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:53.596693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353677e-547e-4fe1-8e83-0e49c1fbe9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:53.597678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353677e-547e-4fe1-8e83-0e49c1fbe9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:53.597966 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353677e-547e-4fe1-8e83-0e49c1fbe9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:53.598246 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353677e-547e-4fe1-8e83-0e49c1fbe9c3 None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:53.599506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353677e-547e-4fe1-8e83-0e49c1fbe9c3 None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:53.600984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353677e-547e-4fe1-8e83-0e49c1fbe9c3 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:53.602723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 96/207] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:53 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:53.608687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9794b-1faf-46b7-88e6-55cf1c63e1e8 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:53.751484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9794b-1faf-46b7-88e6-55cf1c63e1e8 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:53.792765 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8de03-9697-49c6-9942-2513ab105c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:53.793847 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8de03-9697-49c6-9942-2513ab105c86 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:53.794353 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8de03-9697-49c6-9942-2513ab105c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:53.794858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8de03-9697-49c6-9942-2513ab105c86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:53.795916 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8de03-9697-49c6-9942-2513ab105c86 None admin] RBAC: Adding query filter params (role.id=127c304ac7ee4fce8de777dbd4037b59, scope.project.id=2710654d490f43b0adadfd7a62df84bd, user.id=87dd1617d8f84685bd7cbcf2d4ac83ae) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:53.796441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8de03-9697-49c6-9942-2513ab105c86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:53.798439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8de03-9697-49c6-9942-2513ab105c86 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:53.805979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 112/208] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:01:53 2022] GET /identity/v3/role_assignments?user.id=87dd1617d8f84685bd7cbcf2d4ac83ae&scope.project.id=2710654d490f43b0adadfd7a62df84bd&role.id=127c304ac7ee4fce8de777dbd4037b59 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:54.254775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdac8b0-b173-4ec8-b8ab-44bc9e04f55c None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:54.255146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdac8b0-b173-4ec8-b8ab-44bc9e04f55c None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:54.255343 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdac8b0-b173-4ec8-b8ab-44bc9e04f55c None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:54.257060 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 97/209] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:54.266214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95178a-47f9-4ee0-9538-84746476d95d None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:54.267266 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95178a-47f9-4ee0-9538-84746476d95d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:54.267409 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95178a-47f9-4ee0-9538-84746476d95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:54.289431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-2c95178a-47f9-4ee0-9538-84746476d95d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:54.290960 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2c95178a-47f9-4ee0-9538-84746476d95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:54.333188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:54.378348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 113/210] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:54.405787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdac8b0-b173-4ec8-b8ab-44bc9e04f55c None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:54.413620 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afc543-9f65-4a6d-a9c1-ab4d2acb37ed None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:54.413620 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afc543-9f65-4a6d-a9c1-ab4d2acb37ed None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:54.413885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afc543-9f65-4a6d-a9c1-ab4d2acb37ed None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:54.415277 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 114/211] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:54.420522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aade62-2a4c-4cdc-af9d-0eaefbc77bca None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:54.421074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aade62-2a4c-4cdc-af9d-0eaefbc77bca None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:54.421574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aade62-2a4c-4cdc-af9d-0eaefbc77bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:54.441884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-17aade62-2a4c-4cdc-af9d-0eaefbc77bca None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:54.443395 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-17aade62-2a4c-4cdc-af9d-0eaefbc77bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:54.522856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 115/212] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:01:54.541290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdac8b0-b173-4ec8-b8ab-44bc9e04f55c None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:54.548705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aade62-2a4c-4cdc-af9d-0eaefbc77bca None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:54.580171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062190f-be42-430b-a581-313e662ced87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:54.581259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062190f-be42-430b-a581-313e662ced87 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:54.581689 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062190f-be42-430b-a581-313e662ced87 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:54.582136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c062190f-be42-430b-a581-313e662ced87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:54.593989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c062190f-be42-430b-a581-313e662ced87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-c062190f-be42-430b-a581-313e662ced87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:54.603461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:54.604542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:54.605491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:54.605491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:54.605491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:54.605491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:54.605491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:54.605491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:54.605491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:54.605491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:54.605491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:54.605491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:54.605491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:54.605491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:01:54.607479 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 98/213] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:54.617007 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062190f-be42-430b-a581-313e662ced87 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:54.659249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:54.662282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aade62-2a4c-4cdc-af9d-0eaefbc77bca None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:54.671410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:54.699062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ac4ec-791a-45d8-8aad-fa8eaf3ddbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:54.699996 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ac4ec-791a-45d8-8aad-fa8eaf3ddbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:54.700392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ac4ec-791a-45d8-8aad-fa8eaf3ddbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:54.700733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ac4ec-791a-45d8-8aad-fa8eaf3ddbe7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:54.702168 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ac4ec-791a-45d8-8aad-fa8eaf3ddbe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:54.704238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ac4ec-791a-45d8-8aad-fa8eaf3ddbe7 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:54.706387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 116/214] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:01:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:54.712174 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ac4ec-791a-45d8-8aad-fa8eaf3ddbe7 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:54.741900 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c062190f-be42-430b-a581-313e662ced87 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:54.788025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bdeec-6e61-47ef-8bb8-882b7e32be44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:54.788725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bdeec-6e61-47ef-8bb8-882b7e32be44 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:54.789333 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bdeec-6e61-47ef-8bb8-882b7e32be44 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:54.789687 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bdeec-6e61-47ef-8bb8-882b7e32be44 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:54.791248 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bdeec-6e61-47ef-8bb8-882b7e32be44 None admin] RBAC: Adding query filter params (name=member) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:54.791637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bdeec-6e61-47ef-8bb8-882b7e32be44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:54.793177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bdeec-6e61-47ef-8bb8-882b7e32be44 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:54.805798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 99/215] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:01:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:54.812158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bdeec-6e61-47ef-8bb8-882b7e32be44 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:54.838117 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ac4ec-791a-45d8-8aad-fa8eaf3ddbe7 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:54.877098 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa2f1a-1ed6-4371-9735-bd4f226ebd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:54.877723 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa2f1a-1ed6-4371-9735-bd4f226ebd64 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:54.878008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa2f1a-1ed6-4371-9735-bd4f226ebd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:54.878285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa2f1a-1ed6-4371-9735-bd4f226ebd64 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:54.879008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa2f1a-1ed6-4371-9735-bd4f226ebd64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:54.880670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fa2f1a-1ed6-4371-9735-bd4f226ebd64 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:54.926846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bdeec-6e61-47ef-8bb8-882b7e32be44 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:54.935566 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:54.961880 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 117/216] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:01:54 2022] POST /identity/v3/users => generated 297 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:01:54.971700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29071974-8a45-48be-82b1-76ca2e242303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:54.973277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29071974-8a45-48be-82b1-76ca2e242303 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:54.973960 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29071974-8a45-48be-82b1-76ca2e242303 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:54.974445 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29071974-8a45-48be-82b1-76ca2e242303 None admin] PATH_INFO: `/v3/users/5528617edcf64c3987a433eb08a4fd99` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:54.993082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29071974-8a45-48be-82b1-76ca2e242303 None admin] RBAC: Authorizing `identity:get_user(user_id=5528617edcf64c3987a433eb08a4fd99)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:54.995094 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29071974-8a45-48be-82b1-76ca2e242303 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:54.997309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 100/217] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:54 2022] GET /identity/v3/users/5528617edcf64c3987a433eb08a4fd99 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:55.006231 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa2f1a-1ed6-4371-9735-bd4f226ebd64 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:55.037771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:55.127731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa2f1a-1ed6-4371-9735-bd4f226ebd64 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:55.171972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b3465-e060-445b-abbf-ed6f49ed4823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:55.173196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b3465-e060-445b-abbf-ed6f49ed4823 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:55.173481 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b3465-e060-445b-abbf-ed6f49ed4823 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:55.173758 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b3465-e060-445b-abbf-ed6f49ed4823 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:55.205605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a138af-4fd7-4c6f-8a1c-ad8e1a757384 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:55.206721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a138af-4fd7-4c6f-8a1c-ad8e1a757384 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:55.207476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a138af-4fd7-4c6f-8a1c-ad8e1a757384 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:55.209467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 101/218] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:01:55.218007 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e11ed-d3ae-4322-9968-a2e7fa8066c9 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:55.218614 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e11ed-d3ae-4322-9968-a2e7fa8066c9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:55.218955 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e11ed-d3ae-4322-9968-a2e7fa8066c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:55.240346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-d42e11ed-d3ae-4322-9968-a2e7fa8066c9 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:55.241893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d42e11ed-d3ae-4322-9968-a2e7fa8066c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:55.256004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b3465-e060-445b-abbf-ed6f49ed4823 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:55.257544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b3465-e060-445b-abbf-ed6f49ed4823 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:55.289738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:55.328265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 102/219] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:01:55.354048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e11ed-d3ae-4322-9968-a2e7fa8066c9 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:55.383094 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 118/220] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:01:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:01:55.389362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b3465-e060-445b-abbf-ed6f49ed4823 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:55.503455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:55.514200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e11ed-d3ae-4322-9968-a2e7fa8066c9 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:55.519500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b3465-e060-445b-abbf-ed6f49ed4823 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:55.555274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:55.567883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b23d3e-03a6-4f37-83b6-bed0525f38d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:55.568907 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b23d3e-03a6-4f37-83b6-bed0525f38d8 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:55.570862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b23d3e-03a6-4f37-83b6-bed0525f38d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:55.570862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b23d3e-03a6-4f37-83b6-bed0525f38d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:55.571029 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7efec9d-7b6c-4171-9c94-22121c995329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:55.571362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efec9d-7b6c-4171-9c94-22121c995329 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:55.571820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efec9d-7b6c-4171-9c94-22121c995329 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:55.572283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7efec9d-7b6c-4171-9c94-22121c995329 None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:55.573767 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7efec9d-7b6c-4171-9c94-22121c995329 None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:55.575513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7efec9d-7b6c-4171-9c94-22121c995329 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:55.577611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 119/221] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:55 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:55.579155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e811028-9a2b-4ca4-8a9b-4506618ff8c2 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:55.579877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e811028-9a2b-4ca4-8a9b-4506618ff8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:55.580337 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e811028-9a2b-4ca4-8a9b-4506618ff8c2 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:55.581623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b23d3e-03a6-4f37-83b6-bed0525f38d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:55.582086 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 120/222] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:55.583687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e811028-9a2b-4ca4-8a9b-4506618ff8c2 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-b7b23d3e-03a6-4f37-83b6-bed0525f38d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:55.590879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:55.591793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:55.592578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:55.592578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:55.592578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:55.592578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:55.592578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:55.592578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:55.592578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:55.592578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:55.592578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:55.592578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:55.592578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:55.592578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:01:55.592578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 103/223] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:01:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:01:55.597300 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f58200-ad36-4430-9920-ad49a6d5fbd7 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:55.597300 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f58200-ad36-4430-9920-ad49a6d5fbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:55.597300 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f58200-ad36-4430-9920-ad49a6d5fbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:55.620313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a3f58200-ad36-4430-9920-ad49a6d5fbd7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:55.621562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a3f58200-ad36-4430-9920-ad49a6d5fbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:55.710634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e811028-9a2b-4ca4-8a9b-4506618ff8c2 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:55.720325 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 104/224] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:55.725920 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f58200-ad36-4430-9920-ad49a6d5fbd7 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:55.771300 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c175138-acbe-46a1-93a9-fa40acc484f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:55.772209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c175138-acbe-46a1-93a9-fa40acc484f2 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:55.772644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c175138-acbe-46a1-93a9-fa40acc484f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:55.773100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c175138-acbe-46a1-93a9-fa40acc484f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:55.774160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c175138-acbe-46a1-93a9-fa40acc484f2 None admin] RBAC: Adding query filter params (role.id=c91413ec3feb454ca7186eb52a57ed96, scope.project.id=354b4a7dec454bbe9294b154303e7f7e, user.id=5528617edcf64c3987a433eb08a4fd99) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:55.774675 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c175138-acbe-46a1-93a9-fa40acc484f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:55.776347 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c175138-acbe-46a1-93a9-fa40acc484f2 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:55.783278 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 121/225] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:01:55 2022] GET /identity/v3/role_assignments?user.id=5528617edcf64c3987a433eb08a4fd99&scope.project.id=354b4a7dec454bbe9294b154303e7f7e&role.id=c91413ec3feb454ca7186eb52a57ed96 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:55.784379 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c175138-acbe-46a1-93a9-fa40acc484f2 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:55.825086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:55.845410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f58200-ad36-4430-9920-ad49a6d5fbd7 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:55.850808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:55.903166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c175138-acbe-46a1-93a9-fa40acc484f2 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:55.920857 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923428df-9822-40aa-a27e-8eca6a3c3d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:55.921555 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923428df-9822-40aa-a27e-8eca6a3c3d29 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:55.921845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923428df-9822-40aa-a27e-8eca6a3c3d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:55.922128 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923428df-9822-40aa-a27e-8eca6a3c3d29 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:55.923601 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923428df-9822-40aa-a27e-8eca6a3c3d29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:55.923858 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923428df-9822-40aa-a27e-8eca6a3c3d29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:55.925333 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923428df-9822-40aa-a27e-8eca6a3c3d29 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:55.942751 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 105/226] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:55.948666 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a52212-ce7c-4283-b722-a863332730a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:55.950010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923428df-9822-40aa-a27e-8eca6a3c3d29 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:55.950227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a52212-ce7c-4283-b722-a863332730a0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:55.950806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a52212-ce7c-4283-b722-a863332730a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:55.951366 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a52212-ce7c-4283-b722-a863332730a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:55.964927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a52212-ce7c-4283-b722-a863332730a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-c6a52212-ce7c-4283-b722-a863332730a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:55.974280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:55.975249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:55.976015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:55.976015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:55.976015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:55.976015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:55.976015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:55.976015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:55.976015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:55.976015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:55.976015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:55.976015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:55.976015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:01:55.976015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:01:55.978565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 122/227] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:01:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:55.986692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a52212-ce7c-4283-b722-a863332730a0 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:56.139332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923428df-9822-40aa-a27e-8eca6a3c3d29 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:56.161340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a52212-ce7c-4283-b722-a863332730a0 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:56.191419 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:56.209757 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0105d3-96de-475f-bcbe-2572964f8556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:56.210646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0105d3-96de-475f-bcbe-2572964f8556 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:56.210965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0105d3-96de-475f-bcbe-2572964f8556 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:56.211238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0105d3-96de-475f-bcbe-2572964f8556 None admin] PATH_INFO: `/v3/users/c1642bebaf594611827900f344ef5b74` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:56.213134 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0105d3-96de-475f-bcbe-2572964f8556 None admin] RBAC: Authorizing `identity:get_user(user_id=c1642bebaf594611827900f344ef5b74)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:56.214587 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0105d3-96de-475f-bcbe-2572964f8556 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:56.218773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 106/228] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:55 2022] GET /identity/v3/users/c1642bebaf594611827900f344ef5b74 => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:56.223711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0105d3-96de-475f-bcbe-2572964f8556 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:56.228261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70999d-7d67-4600-b9d4-f3cbd2f22f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:56.228943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70999d-7d67-4600-b9d4-f3cbd2f22f87 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:56.229248 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70999d-7d67-4600-b9d4-f3cbd2f22f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:56.229528 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70999d-7d67-4600-b9d4-f3cbd2f22f87 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:56.230285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70999d-7d67-4600-b9d4-f3cbd2f22f87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:56.230591 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70999d-7d67-4600-b9d4-f3cbd2f22f87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:56.232039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70999d-7d67-4600-b9d4-f3cbd2f22f87 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:56.252982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 123/229] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:01:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:56.260546 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70999d-7d67-4600-b9d4-f3cbd2f22f87 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:56.261142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:56.400987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0105d3-96de-475f-bcbe-2572964f8556 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:56.434966 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cdd89-e88a-4fd6-8d24-cf7c5ffa7921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:56.436051 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cdd89-e88a-4fd6-8d24-cf7c5ffa7921 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:56.441048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cdd89-e88a-4fd6-8d24-cf7c5ffa7921 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:56.441048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cdd89-e88a-4fd6-8d24-cf7c5ffa7921 None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:56.441048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cdd89-e88a-4fd6-8d24-cf7c5ffa7921 None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:56.441048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cdd89-e88a-4fd6-8d24-cf7c5ffa7921 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:56.441384 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 107/230] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:56 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:56.446825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cdd89-e88a-4fd6-8d24-cf7c5ffa7921 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:56.475096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70999d-7d67-4600-b9d4-f3cbd2f22f87 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:56.573370 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:56.585585 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cdd89-e88a-4fd6-8d24-cf7c5ffa7921 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:56.615803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449139d-1166-4eb7-9238-df80b50caaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:56.616712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449139d-1166-4eb7-9238-df80b50caaf3 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:56.617037 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449139d-1166-4eb7-9238-df80b50caaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:56.617322 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449139d-1166-4eb7-9238-df80b50caaf3 None admin] PATH_INFO: `/v3/users/c1642bebaf594611827900f344ef5b74` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:56.618565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449139d-1166-4eb7-9238-df80b50caaf3 None admin] RBAC: Authorizing `identity:get_user(user_id=c1642bebaf594611827900f344ef5b74)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:56.620062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449139d-1166-4eb7-9238-df80b50caaf3 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:56.621916 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 124/231] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:56 2022] GET /identity/v3/users/c1642bebaf594611827900f344ef5b74 => generated 283 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:56.628178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449139d-1166-4eb7-9238-df80b50caaf3 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:56.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3207899-4e42-42a2-8f89-7af77a15fa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:56.632027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3207899-4e42-42a2-8f89-7af77a15fa98 None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:56.632318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3207899-4e42-42a2-8f89-7af77a15fa98 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:56.632599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3207899-4e42-42a2-8f89-7af77a15fa98 None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e/users/c1642bebaf594611827900f344ef5b74/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:56.637392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3207899-4e42-42a2-8f89-7af77a15fa98 None admin] RBAC: Authorizing `identity:create_grant(project_id=354b4a7dec454bbe9294b154303e7f7e, user_id=c1642bebaf594611827900f344ef5b74, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:56.638860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3207899-4e42-42a2-8f89-7af77a15fa98 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:56.688143 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:56.713071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 108/232] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:01:56 2022] PUT /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e/users/c1642bebaf594611827900f344ef5b74/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:01:56.764336 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449139d-1166-4eb7-9238-df80b50caaf3 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:56.803107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd01e28-05d6-41d4-a693-70cdda18b14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:56.804013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd01e28-05d6-41d4-a693-70cdda18b14d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:56.804366 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd01e28-05d6-41d4-a693-70cdda18b14d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:56.804676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd01e28-05d6-41d4-a693-70cdda18b14d None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:56.806021 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd01e28-05d6-41d4-a693-70cdda18b14d None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:56.807590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd01e28-05d6-41d4-a693-70cdda18b14d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:56.809813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 125/233] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:56 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:56.816338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3207899-4e42-42a2-8f89-7af77a15fa98 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:56.944098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3207899-4e42-42a2-8f89-7af77a15fa98 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:56.985550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d56c7-671a-4a34-9da6-3f6c856c91ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:56.986568 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d56c7-671a-4a34-9da6-3f6c856c91ff None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:56.986886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d56c7-671a-4a34-9da6-3f6c856c91ff None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:56.987182 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d56c7-671a-4a34-9da6-3f6c856c91ff None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e/users/c1642bebaf594611827900f344ef5b74/roles/d241ebe8f5654f46a5a92be6f6b0cc36` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:56.989563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d56c7-671a-4a34-9da6-3f6c856c91ff None admin] RBAC: Authorizing `identity:create_grant(project_id=354b4a7dec454bbe9294b154303e7f7e, user_id=c1642bebaf594611827900f344ef5b74, role_id=d241ebe8f5654f46a5a92be6f6b0cc36)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:56.991033 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d56c7-671a-4a34-9da6-3f6c856c91ff None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:57.107970 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 109/234] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:01:56 2022] PUT /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e/users/c1642bebaf594611827900f344ef5b74/roles/d241ebe8f5654f46a5a92be6f6b0cc36 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:01:57.862453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd178881-e755-4655-93c2-6c215ddc92be None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:57.862959 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd178881-e755-4655-93c2-6c215ddc92be None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:57.862959 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd178881-e755-4655-93c2-6c215ddc92be None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:57.864283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 126/235] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:57.879050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783837f-ec33-400b-ac17-63d5b3a313b3 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:57.879553 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783837f-ec33-400b-ac17-63d5b3a313b3 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:57.879625 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783837f-ec33-400b-ac17-63d5b3a313b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:57.884727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:57.899219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-4783837f-ec33-400b-ac17-63d5b3a313b3 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:57.900777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4783837f-ec33-400b-ac17-63d5b3a313b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:57.917290 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c679ccc-ac09-4038-a29c-8b60366ee005 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:57.917608 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c679ccc-ac09-4038-a29c-8b60366ee005 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:57.917853 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c679ccc-ac09-4038-a29c-8b60366ee005 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:57.919244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 127/236] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:57.926310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994445af-e621-4ddb-886a-f83ed6444bce None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:57.926613 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994445af-e621-4ddb-886a-f83ed6444bce None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:57.926894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994445af-e621-4ddb-886a-f83ed6444bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:57.951848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-994445af-e621-4ddb-886a-f83ed6444bce None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:57.953098 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-994445af-e621-4ddb-886a-f83ed6444bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:58.041515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 110/237] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:58.065594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783837f-ec33-400b-ac17-63d5b3a313b3 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:58.065910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 128/238] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:01:58.092487 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994445af-e621-4ddb-886a-f83ed6444bce None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:58.144015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:58.211062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783837f-ec33-400b-ac17-63d5b3a313b3 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:58.253768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994445af-e621-4ddb-886a-f83ed6444bce None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:58.285038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:58.303113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21525726-e081-4bc8-82fb-30c37d90282b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:58.304363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21525726-e081-4bc8-82fb-30c37d90282b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:58.304754 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21525726-e081-4bc8-82fb-30c37d90282b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:58.305294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21525726-e081-4bc8-82fb-30c37d90282b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:58.313132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56aae1-11cc-4174-98ab-24526d71ca30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:58.314569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56aae1-11cc-4174-98ab-24526d71ca30 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:58.314872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56aae1-11cc-4174-98ab-24526d71ca30 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:58.315151 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56aae1-11cc-4174-98ab-24526d71ca30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:58.316461 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56aae1-11cc-4174-98ab-24526d71ca30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:58.318005 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56aae1-11cc-4174-98ab-24526d71ca30 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:58.319984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 129/239] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:01:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:58.325536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56aae1-11cc-4174-98ab-24526d71ca30 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:58.333594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21525726-e081-4bc8-82fb-30c37d90282b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-21525726-e081-4bc8-82fb-30c37d90282b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:58.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:58.345473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:58.346241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:58.346241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:58.346241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:58.346241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:58.346241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:58.346241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:58.346241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:58.346241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:58.346241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:58.346241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:58.346241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:01:58.346241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:01:58.351170 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 111/240] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:58.352658 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511697dd-ee86-44de-b387-4456e4d95b46 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:58.352950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511697dd-ee86-44de-b387-4456e4d95b46 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:58.353341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511697dd-ee86-44de-b387-4456e4d95b46 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:58.354834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 112/241] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:58.355656 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511697dd-ee86-44de-b387-4456e4d95b46 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:58.436175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56aae1-11cc-4174-98ab-24526d71ca30 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:58.450799 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:58.474692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511697dd-ee86-44de-b387-4456e4d95b46 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:58.476059 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e77bea-0f47-4136-937b-5440f4e9d778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:58.476891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e77bea-0f47-4136-937b-5440f4e9d778 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:58.477363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e77bea-0f47-4136-937b-5440f4e9d778 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:58.477711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e77bea-0f47-4136-937b-5440f4e9d778 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:58.478504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e77bea-0f47-4136-937b-5440f4e9d778 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:58.480046 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e77bea-0f47-4136-937b-5440f4e9d778 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:58.511904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f074fb-234e-4c2d-b9ec-f416af806594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:58.512555 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f074fb-234e-4c2d-b9ec-f416af806594 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:58.512834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f074fb-234e-4c2d-b9ec-f416af806594 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:58.513140 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f074fb-234e-4c2d-b9ec-f416af806594 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:58.513897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f074fb-234e-4c2d-b9ec-f416af806594 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:58.514205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f074fb-234e-4c2d-b9ec-f416af806594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:58.515606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f074fb-234e-4c2d-b9ec-f416af806594 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:58.527243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 113/242] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:01:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:58.529445 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b173b7d-42f4-4910-82e9-27a1f2e93426 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:58.529922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b173b7d-42f4-4910-82e9-27a1f2e93426 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:58.530389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b173b7d-42f4-4910-82e9-27a1f2e93426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:58.553899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 130/243] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:01:58 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:01:58.553899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e77bea-0f47-4136-937b-5440f4e9d778 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:58.572211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-4b173b7d-42f4-4910-82e9-27a1f2e93426 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:58.573471 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4b173b7d-42f4-4910-82e9-27a1f2e93426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:58.685195 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:58.728863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 114/244] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:58.760416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b173b7d-42f4-4910-82e9-27a1f2e93426 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:58.778063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e77bea-0f47-4136-937b-5440f4e9d778 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:58.838902 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6f651-1b0a-49b3-8d7d-f85f2aa83e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:58.839801 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6f651-1b0a-49b3-8d7d-f85f2aa83e00 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:58.840085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6f651-1b0a-49b3-8d7d-f85f2aa83e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:58.840361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6f651-1b0a-49b3-8d7d-f85f2aa83e00 None admin] PATH_INFO: `/v3/users/1226b997c5a04981bf3f30a2f71e49a1` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:58.856893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6f651-1b0a-49b3-8d7d-f85f2aa83e00 None admin] RBAC: Authorizing `identity:get_user(user_id=1226b997c5a04981bf3f30a2f71e49a1)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:58.858456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6f651-1b0a-49b3-8d7d-f85f2aa83e00 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:58.861065 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 131/245] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:58 2022] GET /identity/v3/users/1226b997c5a04981bf3f30a2f71e49a1 => generated 297 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:58.866372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6f651-1b0a-49b3-8d7d-f85f2aa83e00 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:58.895609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:58.904567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b173b7d-42f4-4910-82e9-27a1f2e93426 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:58.947565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbdf40-997b-4e07-a09b-23e17f6dd782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:58.949399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbdf40-997b-4e07-a09b-23e17f6dd782 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:58.949869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbdf40-997b-4e07-a09b-23e17f6dd782 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:58.950306 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bbdf40-997b-4e07-a09b-23e17f6dd782 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:58.964798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bbdf40-997b-4e07-a09b-23e17f6dd782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:58.975321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6f651-1b0a-49b3-8d7d-f85f2aa83e00 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-42bbdf40-997b-4e07-a09b-23e17f6dd782 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:58.975869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:58.976741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:58.977546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:58.977546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:58.977546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:58.977546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:58.977546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:58.977546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:58.977546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:58.977546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:58.977546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:58.977546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:58.977546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:01:58.977546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:01:58.979877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 115/246] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:58.986628 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbdf40-997b-4e07-a09b-23e17f6dd782 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:59.014220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df224279-c266-4784-98b5-b46f24650ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:59.014996 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df224279-c266-4784-98b5-b46f24650ecf None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:59.015348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df224279-c266-4784-98b5-b46f24650ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:59.015685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df224279-c266-4784-98b5-b46f24650ecf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:59.053238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:59.098206 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df224279-c266-4784-98b5-b46f24650ecf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:59.099704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df224279-c266-4784-98b5-b46f24650ecf None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:59.114424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bbdf40-997b-4e07-a09b-23e17f6dd782 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:59.128685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:59.153414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0db122-ea0d-431f-8806-4c8c9276b9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:59.154529 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0db122-ea0d-431f-8806-4c8c9276b9bc None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:59.155009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0db122-ea0d-431f-8806-4c8c9276b9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:59.155473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0db122-ea0d-431f-8806-4c8c9276b9bc None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:59.156430 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0db122-ea0d-431f-8806-4c8c9276b9bc None admin] RBAC: Adding query filter params (name=member) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:59.157604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0db122-ea0d-431f-8806-4c8c9276b9bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:59.159345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0db122-ea0d-431f-8806-4c8c9276b9bc None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:59.170311 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 116/247] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:01:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:59.177339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0db122-ea0d-431f-8806-4c8c9276b9bc None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:59.212232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 132/248] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:01:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:01:59.218267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df224279-c266-4784-98b5-b46f24650ecf None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:59.311563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0db122-ea0d-431f-8806-4c8c9276b9bc None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:59.320845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:59.358969 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a41796-feb0-41f9-9f66-5a26ffb2a58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:59.360590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a41796-feb0-41f9-9f66-5a26ffb2a58e None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:59.361426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a41796-feb0-41f9-9f66-5a26ffb2a58e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:59.361426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a41796-feb0-41f9-9f66-5a26ffb2a58e None admin] PATH_INFO: `/v3/users/5528617edcf64c3987a433eb08a4fd99` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:59.362564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a41796-feb0-41f9-9f66-5a26ffb2a58e None admin] RBAC: Authorizing `identity:get_user(user_id=5528617edcf64c3987a433eb08a4fd99)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:59.364022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a41796-feb0-41f9-9f66-5a26ffb2a58e None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:59.364116 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df224279-c266-4784-98b5-b46f24650ecf None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:59.366663 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 117/249] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:01:59 2022] GET /identity/v3/users/5528617edcf64c3987a433eb08a4fd99 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:59.367961 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d91c0-bbef-49b5-a77d-ef74dc801415 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:59.368250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d91c0-bbef-49b5-a77d-ef74dc801415 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:59.368525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d91c0-bbef-49b5-a77d-ef74dc801415 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:59.374196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 118/250] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:01:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:01:59.375509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d91c0-bbef-49b5-a77d-ef74dc801415 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:59.399504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:59.409303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85545d1f-e2c6-4b4e-a2ab-bf3018bc7155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:59.410413 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85545d1f-e2c6-4b4e-a2ab-bf3018bc7155 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:59.410814 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85545d1f-e2c6-4b4e-a2ab-bf3018bc7155 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:59.411189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85545d1f-e2c6-4b4e-a2ab-bf3018bc7155 None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:59.414568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85545d1f-e2c6-4b4e-a2ab-bf3018bc7155 None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:59.414568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85545d1f-e2c6-4b4e-a2ab-bf3018bc7155 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:59.416231 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 133/251] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:59 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:59.417617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f414e3-113a-4790-88a5-199591c99850 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:59.418009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f414e3-113a-4790-88a5-199591c99850 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:59.418390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f414e3-113a-4790-88a5-199591c99850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:59.439483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-62f414e3-113a-4790-88a5-199591c99850 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:01:59.440867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-62f414e3-113a-4790-88a5-199591c99850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:01:59.512562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d91c0-bbef-49b5-a77d-ef74dc801415 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:59.513462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 134/252] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:01:59.517915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f414e3-113a-4790-88a5-199591c99850 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:59.584824 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938d141-2ce3-4654-9f56-e4a36dbf4420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:59.585683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938d141-2ce3-4654-9f56-e4a36dbf4420 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:59.585965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938d141-2ce3-4654-9f56-e4a36dbf4420 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:59.586239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938d141-2ce3-4654-9f56-e4a36dbf4420 None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:59.587482 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938d141-2ce3-4654-9f56-e4a36dbf4420 None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:59.588930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938d141-2ce3-4654-9f56-e4a36dbf4420 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:59.597448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 119/253] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:01:59 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:01:59.598485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938d141-2ce3-4654-9f56-e4a36dbf4420 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:59.645744 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f414e3-113a-4790-88a5-199591c99850 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:59.660849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:59.681921 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef85aca-9c9a-4cd4-83af-aa6649dd1e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:59.682617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef85aca-9c9a-4cd4-83af-aa6649dd1e24 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:59.682900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef85aca-9c9a-4cd4-83af-aa6649dd1e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:59.683183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef85aca-9c9a-4cd4-83af-aa6649dd1e24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:59.684030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef85aca-9c9a-4cd4-83af-aa6649dd1e24 None admin] RBAC: Adding query filter params (role.id=127c304ac7ee4fce8de777dbd4037b59, scope.project.id=354b4a7dec454bbe9294b154303e7f7e, user.id=1226b997c5a04981bf3f30a2f71e49a1) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:01:59.684345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef85aca-9c9a-4cd4-83af-aa6649dd1e24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:01:59.685922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef85aca-9c9a-4cd4-83af-aa6649dd1e24 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:01:59.689317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:01:59.692445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 135/254] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:01:59 2022] GET /identity/v3/role_assignments?user.id=1226b997c5a04981bf3f30a2f71e49a1&scope.project.id=354b4a7dec454bbe9294b154303e7f7e&role.id=127c304ac7ee4fce8de777dbd4037b59 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:01:59.705868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef85aca-9c9a-4cd4-83af-aa6649dd1e24 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:01:59.730108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938d141-2ce3-4654-9f56-e4a36dbf4420 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:01:59.768673 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56d9e3-6695-467a-84e2-f5f9fe842dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:01:59.769551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56d9e3-6695-467a-84e2-f5f9fe842dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:01:59.769834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56d9e3-6695-467a-84e2-f5f9fe842dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:01:59.770113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56d9e3-6695-467a-84e2-f5f9fe842dd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:01:59.782467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56d9e3-6695-467a-84e2-f5f9fe842dd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-7f56d9e3-6695-467a-84e2-f5f9fe842dd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:01:59.790493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:01:59.791444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:01:59.792215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:01:59.792215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:01:59.792215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:01:59.792215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:01:59.792215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:01:59.792215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:01:59.792215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:01:59.792215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:01:59.792215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:01:59.792215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:01:59.792215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:01:59.792215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:01:59.792215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 120/255] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:01:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:01:59.803545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56d9e3-6695-467a-84e2-f5f9fe842dd0 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:00.002638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef85aca-9c9a-4cd4-83af-aa6649dd1e24 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:00.024783 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56d9e3-6695-467a-84e2-f5f9fe842dd0 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:00.029315 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:00.059389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc0fd4-62c4-44dc-a4d4-e524c1980f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:00.061519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc0fd4-62c4-44dc-a4d4-e524c1980f9c None admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:00.064065 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc0fd4-62c4-44dc-a4d4-e524c1980f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:00.064799 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc0fd4-62c4-44dc-a4d4-e524c1980f9c None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e/users/5528617edcf64c3987a433eb08a4fd99/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:00.069697 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc0fd4-62c4-44dc-a4d4-e524c1980f9c None admin] RBAC: Authorizing `identity:create_grant(project_id=354b4a7dec454bbe9294b154303e7f7e, user_id=5528617edcf64c3987a433eb08a4fd99, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:00.072730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc0fd4-62c4-44dc-a4d4-e524c1980f9c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:00.082175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852ce8d9-359b-4816-bdf7-2c1862a2e21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:00.083319 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ce8d9-359b-4816-bdf7-2c1862a2e21f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:00.083852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ce8d9-359b-4816-bdf7-2c1862a2e21f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:00.084326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852ce8d9-359b-4816-bdf7-2c1862a2e21f None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:00.085401 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ce8d9-359b-4816-bdf7-2c1862a2e21f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:00.085910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ce8d9-359b-4816-bdf7-2c1862a2e21f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:00.087553 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852ce8d9-359b-4816-bdf7-2c1862a2e21f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:00.098416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 121/256] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:01:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:00.100240 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214a12d-7949-4a12-af7a-3bc0ed2c03fd None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:00.100759 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214a12d-7949-4a12-af7a-3bc0ed2c03fd None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:00.101309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214a12d-7949-4a12-af7a-3bc0ed2c03fd None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:00.103113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 122/257] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:00.104024 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214a12d-7949-4a12-af7a-3bc0ed2c03fd None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:00.168761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 136/258] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:01:59 2022] PUT /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e/users/5528617edcf64c3987a433eb08a4fd99/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:02:00.170460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d81ccd-f258-464b-9c29-5ecedf0a0d18 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:00.171008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d81ccd-f258-464b-9c29-5ecedf0a0d18 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:00.171620 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d81ccd-f258-464b-9c29-5ecedf0a0d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:00.229192 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-15d81ccd-f258-464b-9c29-5ecedf0a0d18 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:00.230435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-15d81ccd-f258-464b-9c29-5ecedf0a0d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:00.261782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214a12d-7949-4a12-af7a-3bc0ed2c03fd None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:00.337771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4555c0-e9b2-49f5-9976-4b315c15578b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:00.338680 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4555c0-e9b2-49f5-9976-4b315c15578b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:00.338963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4555c0-e9b2-49f5-9976-4b315c15578b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:00.339238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4555c0-e9b2-49f5-9976-4b315c15578b None admin] PATH_INFO: `/v3/users/c1642bebaf594611827900f344ef5b74` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:00.346074 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4555c0-e9b2-49f5-9976-4b315c15578b None admin] RBAC: Authorizing `identity:get_user(user_id=c1642bebaf594611827900f344ef5b74)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:00.347583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4555c0-e9b2-49f5-9976-4b315c15578b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:00.349361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 123/259] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:02:00 2022] GET /identity/v3/users/c1642bebaf594611827900f344ef5b74 => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:00.354832 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4555c0-e9b2-49f5-9976-4b315c15578b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:00.385226 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:00.411579 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:00.437148 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 137/260] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:00.466130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d81ccd-f258-464b-9c29-5ecedf0a0d18 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:00.515149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4555c0-e9b2-49f5-9976-4b315c15578b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:00.606216 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7eeda-73d3-49d4-acec-ec7752cdcc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:00.606885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7eeda-73d3-49d4-acec-ec7752cdcc95 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:00.607170 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7eeda-73d3-49d4-acec-ec7752cdcc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:00.607442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7eeda-73d3-49d4-acec-ec7752cdcc95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:00.633237 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d81ccd-f258-464b-9c29-5ecedf0a0d18 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:00.696585 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9ef36-f2c8-457c-8327-4e0cbc8ae22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:00.697444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9ef36-f2c8-457c-8327-4e0cbc8ae22a None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:00.697728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9ef36-f2c8-457c-8327-4e0cbc8ae22a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:00.698003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9ef36-f2c8-457c-8327-4e0cbc8ae22a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:00.723243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9ef36-f2c8-457c-8327-4e0cbc8ae22a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-28c9ef36-f2c8-457c-8327-4e0cbc8ae22a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:00.732516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:00.733456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:00.734338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:00.734338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:00.734338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:00.734338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:00.734338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:00.734338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:00.734338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:00.734338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:00.734338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:00.734338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:00.734338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:00.734338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:00.741974 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 138/261] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:02:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:00.748168 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9ef36-f2c8-457c-8327-4e0cbc8ae22a None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:00.806259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7eeda-73d3-49d4-acec-ec7752cdcc95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:00.807691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7eeda-73d3-49d4-acec-ec7752cdcc95 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:00.828197 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:00.872371 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:00.943276 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9ef36-f2c8-457c-8327-4e0cbc8ae22a None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:01.002851 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78ae82-4281-413b-9be9-8195cc448ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:01.004303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78ae82-4281-413b-9be9-8195cc448ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:01.004791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78ae82-4281-413b-9be9-8195cc448ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:01.005323 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78ae82-4281-413b-9be9-8195cc448ce6 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:01.006384 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78ae82-4281-413b-9be9-8195cc448ce6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:01.006921 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78ae82-4281-413b-9be9-8195cc448ce6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:01.008517 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78ae82-4281-413b-9be9-8195cc448ce6 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:01.009414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 124/262] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 10:02:01.018046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7eeda-73d3-49d4-acec-ec7752cdcc95 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:01.041050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 139/263] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:01.048427 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78ae82-4281-413b-9be9-8195cc448ce6 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:01.203902 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7eeda-73d3-49d4-acec-ec7752cdcc95 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:01.306221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:01.310533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78ae82-4281-413b-9be9-8195cc448ce6 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:01.341128 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:01.344748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6660650-d0b6-4d85-8f2a-d7fcfa7c736f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:01.345906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6660650-d0b6-4d85-8f2a-d7fcfa7c736f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:01.346017 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6660650-d0b6-4d85-8f2a-d7fcfa7c736f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:01.346338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6660650-d0b6-4d85-8f2a-d7fcfa7c736f None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:01.348834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6660650-d0b6-4d85-8f2a-d7fcfa7c736f None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:01.354454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6660650-d0b6-4d85-8f2a-d7fcfa7c736f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:01.356268 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 125/264] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:01 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:01.363748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6660650-d0b6-4d85-8f2a-d7fcfa7c736f None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:01.402076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97dab6-0863-4655-b5aa-f9c0cc5f47fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:01.402967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97dab6-0863-4655-b5aa-f9c0cc5f47fe None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:01.403256 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97dab6-0863-4655-b5aa-f9c0cc5f47fe None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:01.403541 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97dab6-0863-4655-b5aa-f9c0cc5f47fe None admin] PATH_INFO: `/v3/users/c1642bebaf594611827900f344ef5b74` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:01.405537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97dab6-0863-4655-b5aa-f9c0cc5f47fe None admin] RBAC: Authorizing `identity:get_user(user_id=c1642bebaf594611827900f344ef5b74)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:01.407648 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97dab6-0863-4655-b5aa-f9c0cc5f47fe None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:01.416910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 140/265] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:02:01 2022] GET /identity/v3/users/c1642bebaf594611827900f344ef5b74 => generated 283 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:01.423172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97dab6-0863-4655-b5aa-f9c0cc5f47fe None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:01.646931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6660650-d0b6-4d85-8f2a-d7fcfa7c736f None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:01.669459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97dab6-0863-4655-b5aa-f9c0cc5f47fe None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:01.724370 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b8d8e-4e0a-40db-9685-efc2ea46762a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:01.725062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b8d8e-4e0a-40db-9685-efc2ea46762a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:01.725347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b8d8e-4e0a-40db-9685-efc2ea46762a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:01.725707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b8d8e-4e0a-40db-9685-efc2ea46762a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:01.726579 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b8d8e-4e0a-40db-9685-efc2ea46762a None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=354b4a7dec454bbe9294b154303e7f7e, user.id=c1642bebaf594611827900f344ef5b74) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:01.726870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b8d8e-4e0a-40db-9685-efc2ea46762a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:01.728439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b8d8e-4e0a-40db-9685-efc2ea46762a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:01.733827 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a75292-0eaf-41d5-834c-84d346e2d70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:01.734921 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a75292-0eaf-41d5-834c-84d346e2d70e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:01.735942 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a75292-0eaf-41d5-834c-84d346e2d70e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:01.736439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a75292-0eaf-41d5-834c-84d346e2d70e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:01.745146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 126/266] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:02:01 2022] GET /identity/v3/role_assignments?user.id=c1642bebaf594611827900f344ef5b74&scope.project.id=354b4a7dec454bbe9294b154303e7f7e&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 608 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:01.805060 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:01.856574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a75292-0eaf-41d5-834c-84d346e2d70e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:01.858500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a75292-0eaf-41d5-834c-84d346e2d70e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:01.991168 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 141/267] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:01.997731 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b8d8e-4e0a-40db-9685-efc2ea46762a None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:02.075484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:02.101564 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40515c9e-9104-4880-953c-9a594e767105 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:02.102818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40515c9e-9104-4880-953c-9a594e767105 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:02.103301 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40515c9e-9104-4880-953c-9a594e767105 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:02.106327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 142/268] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:02:02.117445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6ad55-6bf3-4c88-9b23-abde6828981a None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:02.117741 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6ad55-6bf3-4c88-9b23-abde6828981a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:02.117984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6ad55-6bf3-4c88-9b23-abde6828981a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:02.122039 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b8d8e-4e0a-40db-9685-efc2ea46762a None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:02.164986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-c7d6ad55-6bf3-4c88-9b23-abde6828981a None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:02.164986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c7d6ad55-6bf3-4c88-9b23-abde6828981a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:02.166294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1670a6-cf85-46d0-bb78-0db528b5716a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:02.167190 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1670a6-cf85-46d0-bb78-0db528b5716a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:02.167475 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1670a6-cf85-46d0-bb78-0db528b5716a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:02.167749 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1670a6-cf85-46d0-bb78-0db528b5716a None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:02.169054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1670a6-cf85-46d0-bb78-0db528b5716a None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:02.170606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1670a6-cf85-46d0-bb78-0db528b5716a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:02.173215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 127/269] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:01 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:02.181470 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1670a6-cf85-46d0-bb78-0db528b5716a None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:02.304751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:02.317133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 143/270] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:02.318898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d0d5d-f5b1-458f-a4b5-d3f4f391d462 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:02.319170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d0d5d-f5b1-458f-a4b5-d3f4f391d462 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:02.319457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d0d5d-f5b1-458f-a4b5-d3f4f391d462 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:02.321274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 144/271] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:02.332138 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1670a6-cf85-46d0-bb78-0db528b5716a None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:02.339184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc1120-4714-4c20-9b70-b03cbae6d90f None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:02.339478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc1120-4714-4c20-9b70-b03cbae6d90f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:02.339854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fc1120-4714-4c20-9b70-b03cbae6d90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:02.363208 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-c4fc1120-4714-4c20-9b70-b03cbae6d90f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:02.364599 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c4fc1120-4714-4c20-9b70-b03cbae6d90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:02.367992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:02.373517 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc019b-e527-4556-8ae2-bf79aa09252e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:02.374195 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc019b-e527-4556-8ae2-bf79aa09252e None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:02.374482 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc019b-e527-4556-8ae2-bf79aa09252e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:02.374762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc019b-e527-4556-8ae2-bf79aa09252e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:02.375682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc019b-e527-4556-8ae2-bf79aa09252e None admin] RBAC: Adding query filter params (role.id=d241ebe8f5654f46a5a92be6f6b0cc36, scope.project.id=354b4a7dec454bbe9294b154303e7f7e, user.id=c1642bebaf594611827900f344ef5b74) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:02.376467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc019b-e527-4556-8ae2-bf79aa09252e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:02.378133 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc019b-e527-4556-8ae2-bf79aa09252e None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:02.385259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 128/272] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:02:02 2022] GET /identity/v3/role_assignments?user.id=c1642bebaf594611827900f344ef5b74&scope.project.id=354b4a7dec454bbe9294b154303e7f7e&role.id=d241ebe8f5654f46a5a92be6f6b0cc36 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:02.386304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc019b-e527-4556-8ae2-bf79aa09252e None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:02.505543 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 145/273] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:02:02.534188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc1120-4714-4c20-9b70-b03cbae6d90f None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:02.611366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc019b-e527-4556-8ae2-bf79aa09252e None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:02.670674 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce8f13-c857-4263-aa0a-c2d4e5303a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:02.671649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce8f13-c857-4263-aa0a-c2d4e5303a2d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:02.671946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce8f13-c857-4263-aa0a-c2d4e5303a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:02.672315 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce8f13-c857-4263-aa0a-c2d4e5303a2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:02.697006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce8f13-c857-4263-aa0a-c2d4e5303a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-e3ce8f13-c857-4263-aa0a-c2d4e5303a2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:02.708113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:02.709078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:02.709877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:02.709877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:02.709877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:02.709877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:02.709877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:02.709877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:02.709877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:02.709877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:02.709877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:02.709877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:02.709877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:02.709877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:02.712798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 129/274] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:02.719766 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fc1120-4714-4c20-9b70-b03cbae6d90f None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:02.721906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce8f13-c857-4263-aa0a-c2d4e5303a2d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:02.755861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:02.766528 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef8d71-8dca-4bcb-a69a-cb452f1ce513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:02.767216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef8d71-8dca-4bcb-a69a-cb452f1ce513 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:02.767572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef8d71-8dca-4bcb-a69a-cb452f1ce513 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:02.767852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef8d71-8dca-4bcb-a69a-cb452f1ce513 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:02.773065 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef8d71-8dca-4bcb-a69a-cb452f1ce513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:02.773065 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef8d71-8dca-4bcb-a69a-cb452f1ce513 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:02.777679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 146/275] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:02.778597 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef8d71-8dca-4bcb-a69a-cb452f1ce513 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:02.810477 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:02.882316 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce8f13-c857-4263-aa0a-c2d4e5303a2d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:02.907506 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef8d71-8dca-4bcb-a69a-cb452f1ce513 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:02.921612 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be245c1e-ee77-420c-97f4-8c8161348111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:02.922816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be245c1e-ee77-420c-97f4-8c8161348111 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:02.923090 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be245c1e-ee77-420c-97f4-8c8161348111 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:02.923350 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be245c1e-ee77-420c-97f4-8c8161348111 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:02.924202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be245c1e-ee77-420c-97f4-8c8161348111 None admin] RBAC: Adding query filter params (name=member) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:02.924502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be245c1e-ee77-420c-97f4-8c8161348111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:02.925915 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be245c1e-ee77-420c-97f4-8c8161348111 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:02.935115 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 130/276] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:02:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:02.939507 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe7440-dd5b-421b-830d-099647a89f00 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:02.940466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe7440-dd5b-421b-830d-099647a89f00 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:02.941189 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fe7440-dd5b-421b-830d-099647a89f00 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:02.942936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 131/277] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:02.943631 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe7440-dd5b-421b-830d-099647a89f00 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:02.946518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d3919-e42b-4680-9da6-2ccc5fd09833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:02.946518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d3919-e42b-4680-9da6-2ccc5fd09833 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:02.946518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d3919-e42b-4680-9da6-2ccc5fd09833 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:02.946518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d3919-e42b-4680-9da6-2ccc5fd09833 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:02.947277 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d3919-e42b-4680-9da6-2ccc5fd09833 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:02.949702 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d3919-e42b-4680-9da6-2ccc5fd09833 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:03.048713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fe7440-dd5b-421b-830d-099647a89f00 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:03.062039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 147/278] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:02:02 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:02:03.069311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb1af3-8dc0-4f8c-8b7f-1a06ed17d565 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:03.069311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb1af3-8dc0-4f8c-8b7f-1a06ed17d565 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:03.069311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb1af3-8dc0-4f8c-8b7f-1a06ed17d565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:03.081558 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:03.084854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-57cb1af3-8dc0-4f8c-8b7f-1a06ed17d565 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:03.086401 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-57cb1af3-8dc0-4f8c-8b7f-1a06ed17d565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:03.104717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1cd18-de85-4862-b0bf-77223ca52aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:03.106176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1cd18-de85-4862-b0bf-77223ca52aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:03.106507 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1cd18-de85-4862-b0bf-77223ca52aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:03.106787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1cd18-de85-4862-b0bf-77223ca52aa1 None admin] PATH_INFO: `/v3/users/69aaa561e48646e886fca735ff95ef09` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:03.128220 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1cd18-de85-4862-b0bf-77223ca52aa1 None admin] RBAC: Authorizing `identity:get_user(user_id=69aaa561e48646e886fca735ff95ef09)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:03.129782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1cd18-de85-4862-b0bf-77223ca52aa1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:03.133632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 132/279] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:02:02 2022] GET /identity/v3/users/69aaa561e48646e886fca735ff95ef09 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:03.142957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1cd18-de85-4862-b0bf-77223ca52aa1 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:03.158077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:03.189471 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 148/280] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:03.209415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb1af3-8dc0-4f8c-8b7f-1a06ed17d565 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:03.276175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1cd18-de85-4862-b0bf-77223ca52aa1 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:03.321009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a0db7-a899-4a35-a01a-888261196625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:03.321719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a0db7-a899-4a35-a01a-888261196625 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:03.322010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a0db7-a899-4a35-a01a-888261196625 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:03.322378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a0db7-a899-4a35-a01a-888261196625 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:03.323516 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a0db7-a899-4a35-a01a-888261196625 None admin] RBAC: Adding query filter params (role.id=c91413ec3feb454ca7186eb52a57ed96, user.id=69aaa561e48646e886fca735ff95ef09, scope.system=all) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:03.323618 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a0db7-a899-4a35-a01a-888261196625 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:03.325049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a0db7-a899-4a35-a01a-888261196625 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:03.327691 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb1af3-8dc0-4f8c-8b7f-1a06ed17d565 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:03.331103 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 133/281] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Jun 15 10:02:03 2022] GET /identity/v3/role_assignments?user.id=69aaa561e48646e886fca735ff95ef09&scope.system=all&role.id=c91413ec3feb454ca7186eb52a57ed96 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:03.332243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6a039-fc43-44d6-9dac-840e0b8f217c None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:03.332780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6a039-fc43-44d6-9dac-840e0b8f217c None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:03.332780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6a039-fc43-44d6-9dac-840e0b8f217c None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:03.334186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 134/282] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:03.341732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce6358-a168-464f-bdaf-1613d2688ddf None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:03.341732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce6358-a168-464f-bdaf-1613d2688ddf None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:03.341732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce6358-a168-464f-bdaf-1613d2688ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:03.361860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-62ce6358-a168-464f-bdaf-1613d2688ddf None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:03.363130 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-62ce6358-a168-464f-bdaf-1613d2688ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:03.375930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33269273-e150-4665-871f-7ed1905beae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:03.376748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33269273-e150-4665-871f-7ed1905beae2 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:03.377098 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33269273-e150-4665-871f-7ed1905beae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:03.377641 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33269273-e150-4665-871f-7ed1905beae2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:03.393254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33269273-e150-4665-871f-7ed1905beae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-33269273-e150-4665-871f-7ed1905beae2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:03.402287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:03.403252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:03.404020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:03.404020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:03.404020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:03.404020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:03.404020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:03.404020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:03.404020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:03.404020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:03.404020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:03.404020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:03.404020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:02:03.404020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:03.404020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 149/283] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:03.410693 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33269273-e150-4665-871f-7ed1905beae2 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:03.415961 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:03.466882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 135/284] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:02:03.477842 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:03.491112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce6358-a168-464f-bdaf-1613d2688ddf None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:03.561968 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33269273-e150-4665-871f-7ed1905beae2 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:03.618296 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7172ef-4515-4b8d-b765-2894dfd3cb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:03.618988 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7172ef-4515-4b8d-b765-2894dfd3cb36 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:03.619278 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7172ef-4515-4b8d-b765-2894dfd3cb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:03.619559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7172ef-4515-4b8d-b765-2894dfd3cb36 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:03.620331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7172ef-4515-4b8d-b765-2894dfd3cb36 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:03.620815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7172ef-4515-4b8d-b765-2894dfd3cb36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:03.625993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7172ef-4515-4b8d-b765-2894dfd3cb36 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:03.638174 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce6358-a168-464f-bdaf-1613d2688ddf None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:03.649628 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 150/285] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:02:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:03.675082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7172ef-4515-4b8d-b765-2894dfd3cb36 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:03.711099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443a6c7-11d2-49a1-9778-8af429402b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:03.712218 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443a6c7-11d2-49a1-9778-8af429402b37 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:03.712674 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443a6c7-11d2-49a1-9778-8af429402b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:03.713168 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443a6c7-11d2-49a1-9778-8af429402b37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:03.744160 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443a6c7-11d2-49a1-9778-8af429402b37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-6443a6c7-11d2-49a1-9778-8af429402b37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:03.760999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:03.761957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:03.762733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:03.762733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:03.762733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:03.762733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:03.762733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:03.762733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:03.762733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:03.762733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:03.762733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:03.762733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:03.762733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:03.762733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:03.764998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 136/286] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:03.772610 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443a6c7-11d2-49a1-9778-8af429402b37 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:03.798441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7172ef-4515-4b8d-b765-2894dfd3cb36 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:03.801787 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:03.838046 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e5e3c-2a70-41ec-bb0c-3009369fbe21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:03.839452 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e5e3c-2a70-41ec-bb0c-3009369fbe21 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:03.839917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e5e3c-2a70-41ec-bb0c-3009369fbe21 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:03.840411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e5e3c-2a70-41ec-bb0c-3009369fbe21 None admin] PATH_INFO: `/v3/users/1226b997c5a04981bf3f30a2f71e49a1` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:03.842435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e5e3c-2a70-41ec-bb0c-3009369fbe21 None admin] RBAC: Authorizing `identity:get_user(user_id=1226b997c5a04981bf3f30a2f71e49a1)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:03.844803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e5e3c-2a70-41ec-bb0c-3009369fbe21 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:03.846736 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 151/287] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:02:03 2022] GET /identity/v3/users/1226b997c5a04981bf3f30a2f71e49a1 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:03.852326 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e5e3c-2a70-41ec-bb0c-3009369fbe21 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:03.859591 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:03.983662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443a6c7-11d2-49a1-9778-8af429402b37 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:04.023870 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e5e3c-2a70-41ec-bb0c-3009369fbe21 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:04.030396 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d418c7-6abf-4ad9-980d-96aaa59bee61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:04.031076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d418c7-6abf-4ad9-980d-96aaa59bee61 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:04.031407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d418c7-6abf-4ad9-980d-96aaa59bee61 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:04.032450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d418c7-6abf-4ad9-980d-96aaa59bee61 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:04.033278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d418c7-6abf-4ad9-980d-96aaa59bee61 None admin] RBAC: Adding query filter params (name=member) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:04.033591 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d418c7-6abf-4ad9-980d-96aaa59bee61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:04.035106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d418c7-6abf-4ad9-980d-96aaa59bee61 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:04.048920 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 137/288] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:02:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:04.059553 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d418c7-6abf-4ad9-980d-96aaa59bee61 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:04.083339 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e90d6-6731-4557-8166-7df4d6f7b455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:04.084220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e90d6-6731-4557-8166-7df4d6f7b455 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:04.084504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e90d6-6731-4557-8166-7df4d6f7b455 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:04.084773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e90d6-6731-4557-8166-7df4d6f7b455 None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:04.086006 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e90d6-6731-4557-8166-7df4d6f7b455 None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:04.087567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e90d6-6731-4557-8166-7df4d6f7b455 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:04.089659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 152/289] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:03 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:04.095446 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e90d6-6731-4557-8166-7df4d6f7b455 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:04.170738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:04.240895 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:04.255954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d418c7-6abf-4ad9-980d-96aaa59bee61 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:04.283280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e90d6-6731-4557-8166-7df4d6f7b455 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:04.321938 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41be6b-3a59-44bb-9f4f-5ba51867383f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:04.322867 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41be6b-3a59-44bb-9f4f-5ba51867383f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:04.323151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41be6b-3a59-44bb-9f4f-5ba51867383f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:04.323422 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41be6b-3a59-44bb-9f4f-5ba51867383f None admin] PATH_INFO: `/v3/users/5528617edcf64c3987a433eb08a4fd99` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:04.325375 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41be6b-3a59-44bb-9f4f-5ba51867383f None admin] RBAC: Authorizing `identity:get_user(user_id=5528617edcf64c3987a433eb08a4fd99)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:04.326833 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b41be6b-3a59-44bb-9f4f-5ba51867383f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:04.328516 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 138/290] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:02:04 2022] GET /identity/v3/users/5528617edcf64c3987a433eb08a4fd99 => generated 297 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:04.335181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41be6b-3a59-44bb-9f4f-5ba51867383f None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:04.353949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e7dda-8a14-4cdb-a893-c6c6178d198d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:04.354858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e7dda-8a14-4cdb-a893-c6c6178d198d None admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:04.355143 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e7dda-8a14-4cdb-a893-c6c6178d198d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:04.356566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e7dda-8a14-4cdb-a893-c6c6178d198d None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e/users/1226b997c5a04981bf3f30a2f71e49a1/roles/127c304ac7ee4fce8de777dbd4037b59` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:04.360907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e7dda-8a14-4cdb-a893-c6c6178d198d None admin] RBAC: Authorizing `identity:create_grant(project_id=354b4a7dec454bbe9294b154303e7f7e, user_id=1226b997c5a04981bf3f30a2f71e49a1, role_id=127c304ac7ee4fce8de777dbd4037b59)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:04.362407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e7dda-8a14-4cdb-a893-c6c6178d198d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:04.438876 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 153/291] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:02:04 2022] PUT /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e/users/1226b997c5a04981bf3f30a2f71e49a1/roles/127c304ac7ee4fce8de777dbd4037b59 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:02:04.468056 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41be6b-3a59-44bb-9f4f-5ba51867383f None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:04.507143 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9c7a5-36d5-428f-bbf2-35ca76af719b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:04.507724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9c7a5-36d5-428f-bbf2-35ca76af719b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:04.508007 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9c7a5-36d5-428f-bbf2-35ca76af719b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:04.508284 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9c7a5-36d5-428f-bbf2-35ca76af719b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:04.545073 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:04.606218 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9c7a5-36d5-428f-bbf2-35ca76af719b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:04.607753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9c7a5-36d5-428f-bbf2-35ca76af719b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:04.733338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 139/292] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:04.739794 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e7dda-8a14-4cdb-a893-c6c6178d198d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:04.849949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:04.853609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e7dda-8a14-4cdb-a893-c6c6178d198d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:04.900475 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e807fc-3a87-4ec0-b76b-b08d75636bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:04.901444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e807fc-3a87-4ec0-b76b-b08d75636bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:04.901728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e807fc-3a87-4ec0-b76b-b08d75636bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:04.902051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e807fc-3a87-4ec0-b76b-b08d75636bb5 None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:04.903268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e807fc-3a87-4ec0-b76b-b08d75636bb5 None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:04.904743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e807fc-3a87-4ec0-b76b-b08d75636bb5 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:04.913126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9c7a5-36d5-428f-bbf2-35ca76af719b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:04.916302 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 154/293] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:04 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:04.991535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:05.045033 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9c7a5-36d5-428f-bbf2-35ca76af719b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:05.093879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc376e1c-16a0-4da5-9946-7584419b7a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:05.095715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc376e1c-16a0-4da5-9946-7584419b7a02 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:05.096222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc376e1c-16a0-4da5-9946-7584419b7a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:05.096691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc376e1c-16a0-4da5-9946-7584419b7a02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:05.097800 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc376e1c-16a0-4da5-9946-7584419b7a02 None admin] RBAC: Adding query filter params (role.id=c91413ec3feb454ca7186eb52a57ed96, scope.project.id=354b4a7dec454bbe9294b154303e7f7e, user.id=5528617edcf64c3987a433eb08a4fd99) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:05.098315 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc376e1c-16a0-4da5-9946-7584419b7a02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:05.100034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc376e1c-16a0-4da5-9946-7584419b7a02 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:05.110070 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 140/294] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:02:04 2022] GET /identity/v3/role_assignments?user.id=5528617edcf64c3987a433eb08a4fd99&scope.project.id=354b4a7dec454bbe9294b154303e7f7e&role.id=c91413ec3feb454ca7186eb52a57ed96 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:05.733344 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673eca1-fc68-480a-a314-fcf68882857e None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:05.733840 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673eca1-fc68-480a-a314-fcf68882857e None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:05.733912 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673eca1-fc68-480a-a314-fcf68882857e None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:05.735858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 155/295] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:05.741582 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21655e66-d541-485f-b231-89a12d4151c4 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:05.742066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21655e66-d541-485f-b231-89a12d4151c4 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:05.742142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21655e66-d541-485f-b231-89a12d4151c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:05.761529 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-21655e66-d541-485f-b231-89a12d4151c4 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:05.762744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-21655e66-d541-485f-b231-89a12d4151c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:05.846257 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14920eb3-00a6-4e89-8389-3df4a3bd5d7a None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:05.846444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14920eb3-00a6-4e89-8389-3df4a3bd5d7a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:05.846727 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14920eb3-00a6-4e89-8389-3df4a3bd5d7a None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:05.848365 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 156/296] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:02:05.854551 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8a8a2-5094-4a29-b496-5e04adf4a280 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:05.855075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8a8a2-5094-4a29-b496-5e04adf4a280 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:05.855353 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8a8a2-5094-4a29-b496-5e04adf4a280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:05.861912 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 141/297] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:05.878492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-44f8a8a2-5094-4a29-b496-5e04adf4a280 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:05.879971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-44f8a8a2-5094-4a29-b496-5e04adf4a280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:05.887630 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21655e66-d541-485f-b231-89a12d4151c4 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:05.949228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:05.963649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 157/298] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:05.965380 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd61e6-ea76-4c68-8363-f66c803db171 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:05.965861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd61e6-ea76-4c68-8363-f66c803db171 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:05.966306 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd61e6-ea76-4c68-8363-f66c803db171 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:05.967741 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 158/299] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:05.973354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86a604-4f94-4aa9-a902-1c3ab9d0e263 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:05.973899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86a604-4f94-4aa9-a902-1c3ab9d0e263 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:05.974372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86a604-4f94-4aa9-a902-1c3ab9d0e263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:06.001295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-4d86a604-4f94-4aa9-a902-1c3ab9d0e263 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:06.002468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-4d86a604-4f94-4aa9-a902-1c3ab9d0e263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:06.014777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21655e66-d541-485f-b231-89a12d4151c4 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:06.054865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da6a0a-f536-49c7-b247-a26b2d427f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:06.055820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da6a0a-f536-49c7-b247-a26b2d427f56 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:06.056106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da6a0a-f536-49c7-b247-a26b2d427f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:06.056381 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da6a0a-f536-49c7-b247-a26b2d427f56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:06.068464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da6a0a-f536-49c7-b247-a26b2d427f56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-a8da6a0a-f536-49c7-b247-a26b2d427f56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:06.076241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:06.077346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:06.078282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:06.078282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:06.078282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:06.078282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:06.078282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:06.078282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:06.078282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:06.078282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:06.078282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:06.078282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:06.078282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:06.078282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:06.078282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 142/300] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:06.078282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da6a0a-f536-49c7-b247-a26b2d427f56 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:06.082121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 159/301] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:02:06.083904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86a604-4f94-4aa9-a902-1c3ab9d0e263 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:06.113332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:06.194461 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86a604-4f94-4aa9-a902-1c3ab9d0e263 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:06.206261 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da6a0a-f536-49c7-b247-a26b2d427f56 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:06.223175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:06.231620 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7051e-b1aa-4e69-ab68-07265723f062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:06.232294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7051e-b1aa-4e69-ab68-07265723f062 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:06.232574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7051e-b1aa-4e69-ab68-07265723f062 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:06.232848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7051e-b1aa-4e69-ab68-07265723f062 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:06.233633 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7051e-b1aa-4e69-ab68-07265723f062 None admin] RBAC: Adding query filter params (name=member) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:06.233935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7051e-b1aa-4e69-ab68-07265723f062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:06.236168 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7051e-b1aa-4e69-ab68-07265723f062 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:06.245053 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 160/302] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:02:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:06.245678 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7051e-b1aa-4e69-ab68-07265723f062 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:06.246795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550fed9-6c06-4bf1-a651-0e4eb2290fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:06.247594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550fed9-6c06-4bf1-a651-0e4eb2290fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:06.247878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550fed9-6c06-4bf1-a651-0e4eb2290fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:06.248741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550fed9-6c06-4bf1-a651-0e4eb2290fd5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:06.252029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550fed9-6c06-4bf1-a651-0e4eb2290fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:06.253544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550fed9-6c06-4bf1-a651-0e4eb2290fd5 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:06.259973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 143/303] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:06.259973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550fed9-6c06-4bf1-a651-0e4eb2290fd5 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:06.347049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:06.356856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7051e-b1aa-4e69-ab68-07265723f062 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:06.374803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550fed9-6c06-4bf1-a651-0e4eb2290fd5 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:06.395382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e81287-a359-4d7e-b9e4-fa44833e7d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:06.396184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e81287-a359-4d7e-b9e4-fa44833e7d55 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:06.396468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e81287-a359-4d7e-b9e4-fa44833e7d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:06.396745 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e81287-a359-4d7e-b9e4-fa44833e7d55 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:06.408497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e81287-a359-4d7e-b9e4-fa44833e7d55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:06.413982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3582d7-1e4f-44a8-bf95-e6a93285d92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:06.414870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3582d7-1e4f-44a8-bf95-e6a93285d92a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:06.415152 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3582d7-1e4f-44a8-bf95-e6a93285d92a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:06.415430 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3582d7-1e4f-44a8-bf95-e6a93285d92a None admin] PATH_INFO: `/v3/users/69aaa561e48646e886fca735ff95ef09` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-96e81287-a359-4d7e-b9e4-fa44833e7d55 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:06.416286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:06.417140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:06.418007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:06.418007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:06.418007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:06.418007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:06.418007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:06.418007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:06.418007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:06.418007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:06.418007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:06.418007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:06.418007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:02:06.418007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:06.418612 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3582d7-1e4f-44a8-bf95-e6a93285d92a None admin] RBAC: Authorizing `identity:get_user(user_id=69aaa561e48646e886fca735ff95ef09)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:06.418678 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 161/304] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:06.418740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3582d7-1e4f-44a8-bf95-e6a93285d92a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:06.418813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e81287-a359-4d7e-b9e4-fa44833e7d55 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:06.421917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 144/305] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:02:06 2022] GET /identity/v3/users/69aaa561e48646e886fca735ff95ef09 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:06.426676 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3582d7-1e4f-44a8-bf95-e6a93285d92a None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:06.480356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:06.534021 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e81287-a359-4d7e-b9e4-fa44833e7d55 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:06.538619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3582d7-1e4f-44a8-bf95-e6a93285d92a None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:06.573186 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d694e5-1642-4e08-8dc6-93d34af0b1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:06.573859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d694e5-1642-4e08-8dc6-93d34af0b1ba None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:06.574146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d694e5-1642-4e08-8dc6-93d34af0b1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:06.574423 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d694e5-1642-4e08-8dc6-93d34af0b1ba None admin] PATH_INFO: `/v3/groups` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:06.575112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d694e5-1642-4e08-8dc6-93d34af0b1ba None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:06.576558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d694e5-1642-4e08-8dc6-93d34af0b1ba None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:06.578142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f3b22-c170-463a-a4de-6bcd005c8f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:06.579594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3b22-c170-463a-a4de-6bcd005c8f88 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:06.579879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3b22-c170-463a-a4de-6bcd005c8f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:06.580233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3b22-c170-463a-a4de-6bcd005c8f88 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:06.581021 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f3b22-c170-463a-a4de-6bcd005c8f88 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:06.581331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f3b22-c170-463a-a4de-6bcd005c8f88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:06.582689 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f3b22-c170-463a-a4de-6bcd005c8f88 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:06.591415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 145/306] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:02:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:06.592209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f3b22-c170-463a-a4de-6bcd005c8f88 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:06.655249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 162/307] 10.222.0.60 () {60 vars in 1134 bytes} [Wed Jun 15 10:02:06 2022] POST /identity/v3/groups => generated 229 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:02:06.656165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d694e5-1642-4e08-8dc6-93d34af0b1ba None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:06.678040 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:06.714222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f3b22-c170-463a-a4de-6bcd005c8f88 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:06.730525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:06.756016 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f681b0c-d417-4b5b-9444-478a6a7379ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:06.757778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f681b0c-d417-4b5b-9444-478a6a7379ea None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:06.758234 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f681b0c-d417-4b5b-9444-478a6a7379ea None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:06.758772 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f681b0c-d417-4b5b-9444-478a6a7379ea None admin] PATH_INFO: `/v3/system/users/69aaa561e48646e886fca735ff95ef09/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:06.760669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f681b0c-d417-4b5b-9444-478a6a7379ea None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=69aaa561e48646e886fca735ff95ef09, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:06.762332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f681b0c-d417-4b5b-9444-478a6a7379ea None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:06.805764 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d694e5-1642-4e08-8dc6-93d34af0b1ba None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:06.848092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597df25-a179-4cc7-87b1-f2595ba669ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:06.849567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597df25-a179-4cc7-87b1-f2595ba669ed None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:06.850076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597df25-a179-4cc7-87b1-f2595ba669ed None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:06.850568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597df25-a179-4cc7-87b1-f2595ba669ed None admin] PATH_INFO: `/v3/users/0a61db65f4cd4d2094c9e304cc3d9766` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:06.853066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 146/308] 10.222.0.60 () {58 vars in 1334 bytes} [Wed Jun 15 10:02:06 2022] PUT /identity/v3/system/users/69aaa561e48646e886fca735ff95ef09/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:02:06.869665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597df25-a179-4cc7-87b1-f2595ba669ed None admin] RBAC: Authorizing `identity:get_user(user_id=0a61db65f4cd4d2094c9e304cc3d9766)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:06.871215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597df25-a179-4cc7-87b1-f2595ba669ed None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:06.873355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 163/309] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:02:06 2022] GET /identity/v3/users/0a61db65f4cd4d2094c9e304cc3d9766 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:06.879801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f681b0c-d417-4b5b-9444-478a6a7379ea None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:07.000418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f681b0c-d417-4b5b-9444-478a6a7379ea None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:07.037115 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4fffa7-2996-4b5a-baa7-e8ee2aa9a7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:07.037724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fffa7-2996-4b5a-baa7-e8ee2aa9a7cd None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:07.038012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fffa7-2996-4b5a-baa7-e8ee2aa9a7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:07.038290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4fffa7-2996-4b5a-baa7-e8ee2aa9a7cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:07.039142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4fffa7-2996-4b5a-baa7-e8ee2aa9a7cd None admin] RBAC: Adding query filter params (role.id=127c304ac7ee4fce8de777dbd4037b59, user.id=0a61db65f4cd4d2094c9e304cc3d9766, scope.system=all) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:07.039457 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4fffa7-2996-4b5a-baa7-e8ee2aa9a7cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:07.040934 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4fffa7-2996-4b5a-baa7-e8ee2aa9a7cd None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:07.053407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 147/310] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Jun 15 10:02:06 2022] GET /identity/v3/role_assignments?user.id=0a61db65f4cd4d2094c9e304cc3d9766&scope.system=all&role.id=127c304ac7ee4fce8de777dbd4037b59 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:07.281044 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efd9d6-010b-4b83-8461-7aba04e7a6c9 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:07.281418 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efd9d6-010b-4b83-8461-7aba04e7a6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:07.281674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efd9d6-010b-4b83-8461-7aba04e7a6c9 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:07.283331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 164/311] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:07.288851 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec16700-bb7f-4729-bc12-9d7be87e8fa9 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:07.289175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec16700-bb7f-4729-bc12-9d7be87e8fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:07.289463 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec16700-bb7f-4729-bc12-9d7be87e8fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:07.307615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-2ec16700-bb7f-4729-bc12-9d7be87e8fa9 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:07.308817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-2ec16700-bb7f-4729-bc12-9d7be87e8fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:07.350332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:07.388284 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 148/312] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:02:07.411655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efd9d6-010b-4b83-8461-7aba04e7a6c9 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:07.483088 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:07.520044 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efd9d6-010b-4b83-8461-7aba04e7a6c9 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:07.567279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341e3c0-2010-41db-903c-d72c9f8352c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:07.568082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e3c0-2010-41db-903c-d72c9f8352c8 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:07.568364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e3c0-2010-41db-903c-d72c9f8352c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:07.568639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341e3c0-2010-41db-903c-d72c9f8352c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:07.580242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341e3c0-2010-41db-903c-d72c9f8352c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-e341e3c0-2010-41db-903c-d72c9f8352c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:07.588183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:07.589119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:07.589900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:07.589900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:07.589900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:07.589900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:07.589900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:07.589900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:07.589900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:07.589900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:07.589900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:07.589900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:07.589900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:02:07.589900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:07.589900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 165/313] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:07.596199 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec16700-bb7f-4729-bc12-9d7be87e8fa9 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:07.705458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec16700-bb7f-4729-bc12-9d7be87e8fa9 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:07.743602 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2160a8b-4155-4cb2-965b-553d670ed143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:07.744278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2160a8b-4155-4cb2-965b-553d670ed143 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:07.744564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2160a8b-4155-4cb2-965b-553d670ed143 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:07.744837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2160a8b-4155-4cb2-965b-553d670ed143 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:07.745648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2160a8b-4155-4cb2-965b-553d670ed143 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:07.745957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2160a8b-4155-4cb2-965b-553d670ed143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:07.747376 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2160a8b-4155-4cb2-965b-553d670ed143 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:07.756612 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 149/314] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:02:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:07.762578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341e3c0-2010-41db-903c-d72c9f8352c8 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:07.870687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341e3c0-2010-41db-903c-d72c9f8352c8 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:07.903858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:07.913955 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20b611-0516-4633-a643-8143e72bec11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:07.925469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20b611-0516-4633-a643-8143e72bec11 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:07.925469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20b611-0516-4633-a643-8143e72bec11 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:07.925469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20b611-0516-4633-a643-8143e72bec11 None admin] PATH_INFO: `/v3/users/1226b997c5a04981bf3f30a2f71e49a1` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:07.925469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20b611-0516-4633-a643-8143e72bec11 None admin] RBAC: Authorizing `identity:get_user(user_id=1226b997c5a04981bf3f30a2f71e49a1)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:07.925469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20b611-0516-4633-a643-8143e72bec11 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:07.925469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 166/315] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:02:07 2022] GET /identity/v3/users/1226b997c5a04981bf3f30a2f71e49a1 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:07.927200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2160a8b-4155-4cb2-965b-553d670ed143 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:07.962132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:08.039100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2160a8b-4155-4cb2-965b-553d670ed143 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:08.076815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2eb1d-b236-4c3c-a894-b46196c9658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:08.077501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2eb1d-b236-4c3c-a894-b46196c9658c None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:08.077787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2eb1d-b236-4c3c-a894-b46196c9658c None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:08.078066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2eb1d-b236-4c3c-a894-b46196c9658c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:08.152361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2eb1d-b236-4c3c-a894-b46196c9658c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:08.153775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2eb1d-b236-4c3c-a894-b46196c9658c None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:08.189699 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:08.277364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 150/316] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:08.282615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20b611-0516-4633-a643-8143e72bec11 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:08.396236 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20b611-0516-4633-a643-8143e72bec11 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:08.433273 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf65795-45c6-4326-a66e-8a9adfe9d740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:08.434490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf65795-45c6-4326-a66e-8a9adfe9d740 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:08.435014 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf65795-45c6-4326-a66e-8a9adfe9d740 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:08.435487 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf65795-45c6-4326-a66e-8a9adfe9d740 None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:08.436904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf65795-45c6-4326-a66e-8a9adfe9d740 None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:08.439118 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf65795-45c6-4326-a66e-8a9adfe9d740 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:08.441214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 167/317] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:08 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:08.453046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2eb1d-b236-4c3c-a894-b46196c9658c None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:08.565982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2eb1d-b236-4c3c-a894-b46196c9658c None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:08.605298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:08.612000 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34ce53-d087-4ccc-a013-331b7194a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:08.618318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34ce53-d087-4ccc-a013-331b7194a5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:08.618318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34ce53-d087-4ccc-a013-331b7194a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:08.618318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34ce53-d087-4ccc-a013-331b7194a5e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:08.618318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34ce53-d087-4ccc-a013-331b7194a5e9 None admin] RBAC: Adding query filter params (role.id=127c304ac7ee4fce8de777dbd4037b59, scope.project.id=354b4a7dec454bbe9294b154303e7f7e, user.id=1226b997c5a04981bf3f30a2f71e49a1) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:08.618318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34ce53-d087-4ccc-a013-331b7194a5e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:08.618318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34ce53-d087-4ccc-a013-331b7194a5e9 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:08.622524 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 151/318] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:02:08 2022] GET /identity/v3/role_assignments?user.id=1226b997c5a04981bf3f30a2f71e49a1&scope.project.id=354b4a7dec454bbe9294b154303e7f7e&role.id=127c304ac7ee4fce8de777dbd4037b59 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:09.014110 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1be8c-4e16-4f83-b20d-ca481d6c0b1b None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:09.014624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1be8c-4e16-4f83-b20d-ca481d6c0b1b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:09.014697 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1be8c-4e16-4f83-b20d-ca481d6c0b1b None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:09.016231 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 168/319] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:09.021559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963b5af-f5c4-4a54-910d-1d670b018ee9 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:09.021974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963b5af-f5c4-4a54-910d-1d670b018ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:09.022139 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963b5af-f5c4-4a54-910d-1d670b018ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:09.039638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-4963b5af-f5c4-4a54-910d-1d670b018ee9 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:09.040834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4963b5af-f5c4-4a54-910d-1d670b018ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:09.114709 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 152/320] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:02:09.137657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1be8c-4e16-4f83-b20d-ca481d6c0b1b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:09.150599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f22a68-1f93-4ce6-82a1-46734b3b726e None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:09.150918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f22a68-1f93-4ce6-82a1-46734b3b726e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:09.151154 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f22a68-1f93-4ce6-82a1-46734b3b726e None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:09.152559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 153/321] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:02:09.157869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd90ea-6e3f-424b-9771-9e03a7474896 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:09.158144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd90ea-6e3f-424b-9771-9e03a7474896 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:09.158424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdd90ea-6e3f-424b-9771-9e03a7474896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:09.175344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-bbdd90ea-6e3f-424b-9771-9e03a7474896 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:09.181616 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-bbdd90ea-6e3f-424b-9771-9e03a7474896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:09.210681 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:09.253497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1be8c-4e16-4f83-b20d-ca481d6c0b1b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:09.255205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 154/322] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:02:09.279473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd90ea-6e3f-424b-9771-9e03a7474896 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:09.298506 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c9f43-5691-4ac9-9d89-88e6d867e754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:09.299332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c9f43-5691-4ac9-9d89-88e6d867e754 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:09.299618 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c9f43-5691-4ac9-9d89-88e6d867e754 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:09.299901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c9f43-5691-4ac9-9d89-88e6d867e754 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:09.301389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c9f43-5691-4ac9-9d89-88e6d867e754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:09.302856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c9f43-5691-4ac9-9d89-88e6d867e754 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:09.304502 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 169/323] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:09.309911 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c9f43-5691-4ac9-9d89-88e6d867e754 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:09.340757 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:09.384505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdd90ea-6e3f-424b-9771-9e03a7474896 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:09.413594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c9f43-5691-4ac9-9d89-88e6d867e754 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:09.420148 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97691b20-e3e1-47f6-b8b2-c7ae97eda73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:09.421387 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97691b20-e3e1-47f6-b8b2-c7ae97eda73b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:09.421669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97691b20-e3e1-47f6-b8b2-c7ae97eda73b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:09.421972 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97691b20-e3e1-47f6-b8b2-c7ae97eda73b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:09.432893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97691b20-e3e1-47f6-b8b2-c7ae97eda73b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-97691b20-e3e1-47f6-b8b2-c7ae97eda73b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:09.440728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:09.441790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:09.447777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:09.447777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:09.447777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:09.447777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:09.447777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:09.447777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:09.447777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:09.447777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:09.447777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:09.447777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:09.447777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:09.447777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:09.447777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 155/324] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:09.448466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:09.448538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa7a7a-ed13-4149-8ede-f74c7278a184 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:09.448538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa7a7a-ed13-4149-8ede-f74c7278a184 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:09.448538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa7a7a-ed13-4149-8ede-f74c7278a184 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:09.448538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 156/325] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:09.448755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be18e3-1882-4af3-82ea-71c679163e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:09.448755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be18e3-1882-4af3-82ea-71c679163e5a None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:09.449084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be18e3-1882-4af3-82ea-71c679163e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:09.449334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be18e3-1882-4af3-82ea-71c679163e5a None admin] PATH_INFO: `/v3/groups` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:09.449677 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa7a7a-ed13-4149-8ede-f74c7278a184 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:09.449970 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be18e3-1882-4af3-82ea-71c679163e5a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:09.451424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be18e3-1882-4af3-82ea-71c679163e5a None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:09.506469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 170/326] 10.222.0.60 () {60 vars in 1134 bytes} [Wed Jun 15 10:02:09 2022] POST /identity/v3/groups => generated 226 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:02:09.506469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b0d26-d192-4fc4-bde9-bb8e19f7ea30 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:09.506469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b0d26-d192-4fc4-bde9-bb8e19f7ea30 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:09.506469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b0d26-d192-4fc4-bde9-bb8e19f7ea30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:09.528059 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-0f0b0d26-d192-4fc4-bde9-bb8e19f7ea30 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:09.529370 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0f0b0d26-d192-4fc4-bde9-bb8e19f7ea30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:09.584623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa7a7a-ed13-4149-8ede-f74c7278a184 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:09.604228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:09.612674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 171/327] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:09.626038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a7a1a-734d-48a2-abcc-856d8028bf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:09.627056 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a7a1a-734d-48a2-abcc-856d8028bf9b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:09.627522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a7a1a-734d-48a2-abcc-856d8028bf9b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:09.627965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a7a1a-734d-48a2-abcc-856d8028bf9b None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:09.628913 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a7a1a-734d-48a2-abcc-856d8028bf9b None admin] RBAC: Adding query filter params (name=member) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:09.629660 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a7a1a-734d-48a2-abcc-856d8028bf9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:09.631278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a7a1a-734d-48a2-abcc-856d8028bf9b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:09.635182 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b0d26-d192-4fc4-bde9-bb8e19f7ea30 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:09.644235 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 157/328] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:02:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:09.650047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a7a1a-734d-48a2-abcc-856d8028bf9b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:09.756578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b0d26-d192-4fc4-bde9-bb8e19f7ea30 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:09.774520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a7a1a-734d-48a2-abcc-856d8028bf9b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:09.790142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:09.810981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5f742-95eb-41bf-9f00-5f3a94f18dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:09.811780 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5f742-95eb-41bf-9f00-5f3a94f18dfd None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:09.812069 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5f742-95eb-41bf-9f00-5f3a94f18dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:09.812350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a5f742-95eb-41bf-9f00-5f3a94f18dfd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:09.830917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a5f742-95eb-41bf-9f00-5f3a94f18dfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:09.831107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3b00b-f530-4ee6-a982-ebe786050386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:09.831394 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3b00b-f530-4ee6-a982-ebe786050386 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:09.832031 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3b00b-f530-4ee6-a982-ebe786050386 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:09.832031 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3b00b-f530-4ee6-a982-ebe786050386 None admin] PATH_INFO: `/v3/users/69aaa561e48646e886fca735ff95ef09` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:09.834323 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3b00b-f530-4ee6-a982-ebe786050386 None admin] RBAC: Authorizing `identity:get_user(user_id=69aaa561e48646e886fca735ff95ef09)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:09.835814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3b00b-f530-4ee6-a982-ebe786050386 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:09.838022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 158/329] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:02:09 2022] GET /identity/v3/users/69aaa561e48646e886fca735ff95ef09 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-06a5f742-95eb-41bf-9f00-5f3a94f18dfd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:09.839800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:09.841134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:09.842193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:09.842193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:09.842193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:09.842193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:09.842193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:09.842193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:09.842193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:09.842193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:09.842193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:09.842193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:09.842193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:02:09.842193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:09.842193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 172/330] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:09.844670 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3b00b-f530-4ee6-a982-ebe786050386 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:09.848112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5f742-95eb-41bf-9f00-5f3a94f18dfd None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:09.920573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:09.994445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a5f742-95eb-41bf-9f00-5f3a94f18dfd None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:09.999702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3b00b-f530-4ee6-a982-ebe786050386 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:10.036212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c758b0d-5cf5-46ec-be5a-6941af2a0223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:10.036895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c758b0d-5cf5-46ec-be5a-6941af2a0223 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:10.037229 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c758b0d-5cf5-46ec-be5a-6941af2a0223 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:10.037514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c758b0d-5cf5-46ec-be5a-6941af2a0223 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:10.038333 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c758b0d-5cf5-46ec-be5a-6941af2a0223 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:10.038591 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c758b0d-5cf5-46ec-be5a-6941af2a0223 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:10.040018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c758b0d-5cf5-46ec-be5a-6941af2a0223 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:10.046401 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2349d5e-2fff-4f9b-8483-9e6c62633d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:10.048273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2349d5e-2fff-4f9b-8483-9e6c62633d55 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:10.048695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2349d5e-2fff-4f9b-8483-9e6c62633d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:10.049152 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2349d5e-2fff-4f9b-8483-9e6c62633d55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:10.050143 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2349d5e-2fff-4f9b-8483-9e6c62633d55 None admin] RBAC: Adding query filter params (role.id=c91413ec3feb454ca7186eb52a57ed96, user.id=69aaa561e48646e886fca735ff95ef09, scope.system=all) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:10.050602 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2349d5e-2fff-4f9b-8483-9e6c62633d55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:10.051596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 173/331] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:02:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:10.052992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2349d5e-2fff-4f9b-8483-9e6c62633d55 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:10.057040 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c758b0d-5cf5-46ec-be5a-6941af2a0223 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:10.068209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 159/332] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Jun 15 10:02:09 2022] GET /identity/v3/role_assignments?user.id=69aaa561e48646e886fca735ff95ef09&scope.system=all&role.id=c91413ec3feb454ca7186eb52a57ed96 => generated 510 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:10.114356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:10.181376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c758b0d-5cf5-46ec-be5a-6941af2a0223 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:10.224492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04090d2a-d018-4bae-a381-74569b5dafbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:10.226097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04090d2a-d018-4bae-a381-74569b5dafbc None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:10.226599 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04090d2a-d018-4bae-a381-74569b5dafbc None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:10.227077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04090d2a-d018-4bae-a381-74569b5dafbc None admin] PATH_INFO: `/v3/users/0a61db65f4cd4d2094c9e304cc3d9766` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:10.228672 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04090d2a-d018-4bae-a381-74569b5dafbc None admin] RBAC: Authorizing `identity:get_user(user_id=0a61db65f4cd4d2094c9e304cc3d9766)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:10.230410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04090d2a-d018-4bae-a381-74569b5dafbc None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:10.232770 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 174/333] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:02:10 2022] GET /identity/v3/users/0a61db65f4cd4d2094c9e304cc3d9766 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:10.242506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2349d5e-2fff-4f9b-8483-9e6c62633d55 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:10.371218 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2349d5e-2fff-4f9b-8483-9e6c62633d55 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:10.373723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:10.417542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e9efd-43f4-4900-a8cc-9d1fec75765f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:10.418906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e9efd-43f4-4900-a8cc-9d1fec75765f None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:10.419390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e9efd-43f4-4900-a8cc-9d1fec75765f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:10.419853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e9efd-43f4-4900-a8cc-9d1fec75765f None admin] PATH_INFO: `/v3/system/users/0a61db65f4cd4d2094c9e304cc3d9766/roles/127c304ac7ee4fce8de777dbd4037b59` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:10.421610 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e9efd-43f4-4900-a8cc-9d1fec75765f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0a61db65f4cd4d2094c9e304cc3d9766, role_id=127c304ac7ee4fce8de777dbd4037b59)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:10.423488 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e9efd-43f4-4900-a8cc-9d1fec75765f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:10.507778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 160/334] 10.222.0.60 () {58 vars in 1334 bytes} [Wed Jun 15 10:02:10 2022] PUT /identity/v3/system/users/0a61db65f4cd4d2094c9e304cc3d9766/roles/127c304ac7ee4fce8de777dbd4037b59 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:02:12.294980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d89b0-dce5-4d1a-a769-1ec49e67ce6a None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:12.295493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d89b0-dce5-4d1a-a769-1ec49e67ce6a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:12.295493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d89b0-dce5-4d1a-a769-1ec49e67ce6a None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:12.297324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 175/335] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:12.305838 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3527ca-f987-44d7-829b-d34bc40d08e5 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:12.306300 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3527ca-f987-44d7-829b-d34bc40d08e5 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:12.306372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3527ca-f987-44d7-829b-d34bc40d08e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:12.325775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-eb3527ca-f987-44d7-829b-d34bc40d08e5 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:12.326967 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-eb3527ca-f987-44d7-829b-d34bc40d08e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:12.400936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 161/336] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:02:12.426432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d89b0-dce5-4d1a-a769-1ec49e67ce6a None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:12.574363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d89b0-dce5-4d1a-a769-1ec49e67ce6a None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:12.596009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:12.611597 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fc5ac-4746-4b93-b66c-38568a155d08 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:12.611981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fc5ac-4746-4b93-b66c-38568a155d08 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:12.612268 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fc5ac-4746-4b93-b66c-38568a155d08 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:12.613969 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 162/337] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:12.619356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fba9a-5d8b-4e04-a651-1ebade7d9a6c None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:12.619937 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fba9a-5d8b-4e04-a651-1ebade7d9a6c None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:12.620426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96fba9a-5d8b-4e04-a651-1ebade7d9a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:12.636561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b2c6f-f002-4dd1-86dc-ee52eed0ac71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:12.637417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b2c6f-f002-4dd1-86dc-ee52eed0ac71 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:12.637712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b2c6f-f002-4dd1-86dc-ee52eed0ac71 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:12.637997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b2c6f-f002-4dd1-86dc-ee52eed0ac71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:12.649657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-c96fba9a-5d8b-4e04-a651-1ebade7d9a6c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:12.651261 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c96fba9a-5d8b-4e04-a651-1ebade7d9a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:12.666303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b2c6f-f002-4dd1-86dc-ee52eed0ac71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-fd9b2c6f-f002-4dd1-86dc-ee52eed0ac71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:12.679274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:12.680188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:12.681170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:12.681170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:12.681170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:12.681170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:12.681170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:12.681170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:12.681170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:12.681170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:12.681170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:12.681170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:12.681170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:12.681170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:12.681170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 176/338] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:02:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:12.685611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5993d8-41ef-4722-9dc0-8bf43f1b1fca None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:12.685611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5993d8-41ef-4722-9dc0-8bf43f1b1fca None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:12.685611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5993d8-41ef-4722-9dc0-8bf43f1b1fca None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:12.685611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 177/339] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:12.689723 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3321b9-4094-45e6-8d92-8cb074067f4f None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:12.689947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3321b9-4094-45e6-8d92-8cb074067f4f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:12.690199 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3321b9-4094-45e6-8d92-8cb074067f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:12.715159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-bd3321b9-4094-45e6-8d92-8cb074067f4f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:12.716394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-bd3321b9-4094-45e6-8d92-8cb074067f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:12.749443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 163/340] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:12.750718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fba9a-5d8b-4e04-a651-1ebade7d9a6c None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:12.796910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:12.844691 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 178/341] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:02:12.845481 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3321b9-4094-45e6-8d92-8cb074067f4f None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:12.938866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96fba9a-5d8b-4e04-a651-1ebade7d9a6c None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:12.982186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121175ca-b3e0-4b5e-bdd8-a8fd90ad43c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:12.982865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121175ca-b3e0-4b5e-bdd8-a8fd90ad43c8 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:12.983245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121175ca-b3e0-4b5e-bdd8-a8fd90ad43c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:12.983525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121175ca-b3e0-4b5e-bdd8-a8fd90ad43c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:12.984652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121175ca-b3e0-4b5e-bdd8-a8fd90ad43c8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:12.985541 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121175ca-b3e0-4b5e-bdd8-a8fd90ad43c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:12.986944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121175ca-b3e0-4b5e-bdd8-a8fd90ad43c8 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:12.995678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 164/342] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:12.996500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121175ca-b3e0-4b5e-bdd8-a8fd90ad43c8 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:13.028022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3321b9-4094-45e6-8d92-8cb074067f4f None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:13.029967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:13.128248 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:13.133716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396b5f6-bb3d-4bc6-afcc-d71c4ad66477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:13.134581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396b5f6-bb3d-4bc6-afcc-d71c4ad66477 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:13.134864 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396b5f6-bb3d-4bc6-afcc-d71c4ad66477 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:13.135163 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396b5f6-bb3d-4bc6-afcc-d71c4ad66477 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:13.136169 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121175ca-b3e0-4b5e-bdd8-a8fd90ad43c8 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:13.179985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5916a-ad4b-4c17-a627-69236ccca0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:13.181015 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5916a-ad4b-4c17-a627-69236ccca0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:13.181389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5916a-ad4b-4c17-a627-69236ccca0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:13.181749 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5916a-ad4b-4c17-a627-69236ccca0f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:13.185611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396b5f6-bb3d-4bc6-afcc-d71c4ad66477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-6396b5f6-bb3d-4bc6-afcc-d71c4ad66477 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:13.194246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:13.195147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:13.195922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:13.195922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:13.195922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:13.195922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:13.195922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:13.195922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:13.195922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:13.195922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:13.195922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:13.195922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:13.195922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:13.195922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:13.195922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 179/343] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:13.200994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396b5f6-bb3d-4bc6-afcc-d71c4ad66477 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:13.201313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5916a-ad4b-4c17-a627-69236ccca0f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-01f5916a-ad4b-4c17-a627-69236ccca0f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:13.210795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:13.211771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:13.212562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:13.212562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:13.212562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:13.212562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:13.212562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:13.212562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:13.212562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:13.212562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:13.212562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:13.212562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:13.212562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:13.212562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:13.212562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 165/344] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:13.218343 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5916a-ad4b-4c17-a627-69236ccca0f6 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:13.435845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396b5f6-bb3d-4bc6-afcc-d71c4ad66477 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:13.450651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5916a-ad4b-4c17-a627-69236ccca0f6 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:13.512209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf10d7-f3dd-44a8-8758-430f260a44bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:13.513283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf10d7-f3dd-44a8-8758-430f260a44bd None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:13.513764 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf10d7-f3dd-44a8-8758-430f260a44bd None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:13.514416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf10d7-f3dd-44a8-8758-430f260a44bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:13.530309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182dba0-42cc-44af-bfce-ebe1216a4175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:13.534827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182dba0-42cc-44af-bfce-ebe1216a4175 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:13.535436 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182dba0-42cc-44af-bfce-ebe1216a4175 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:13.535911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182dba0-42cc-44af-bfce-ebe1216a4175 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:13.537126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182dba0-42cc-44af-bfce-ebe1216a4175 None admin] RBAC: Adding query filter params (name=member) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:13.537642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182dba0-42cc-44af-bfce-ebe1216a4175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:13.541505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182dba0-42cc-44af-bfce-ebe1216a4175 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:13.558854 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 166/345] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:02:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:13.559721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182dba0-42cc-44af-bfce-ebe1216a4175 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:13.587906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:13.612036 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:13.670644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf10d7-f3dd-44a8-8758-430f260a44bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:13.676987 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf10d7-f3dd-44a8-8758-430f260a44bd None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:13.729487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182dba0-42cc-44af-bfce-ebe1216a4175 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:13.780429 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 180/346] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:13.782844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62206f6-1220-4176-9c69-7d30c65cd657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:13.783590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf10d7-f3dd-44a8-8758-430f260a44bd None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:13.783962 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62206f6-1220-4176-9c69-7d30c65cd657 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:13.784420 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62206f6-1220-4176-9c69-7d30c65cd657 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:13.784865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62206f6-1220-4176-9c69-7d30c65cd657 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:13.785814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62206f6-1220-4176-9c69-7d30c65cd657 None admin] RBAC: Adding query filter params (name=member) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:13.786274 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62206f6-1220-4176-9c69-7d30c65cd657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:13.788158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62206f6-1220-4176-9c69-7d30c65cd657 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:13.802221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 167/347] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:02:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:13.803348 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62206f6-1220-4176-9c69-7d30c65cd657 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:13.860519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:13.910856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf10d7-f3dd-44a8-8758-430f260a44bd None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:13.923434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:13.948135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a01a61-ed15-407a-a405-0a91d53acb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:13.948822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a01a61-ed15-407a-a405-0a91d53acb6d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:13.949135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a01a61-ed15-407a-a405-0a91d53acb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:13.949464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a01a61-ed15-407a-a405-0a91d53acb6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:13.958061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62206f6-1220-4176-9c69-7d30c65cd657 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:14.015725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07b3e9-e78a-4569-a180-e752c3e0c34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:14.016639 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07b3e9-e78a-4569-a180-e752c3e0c34e None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:14.017053 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07b3e9-e78a-4569-a180-e752c3e0c34e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:14.017336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07b3e9-e78a-4569-a180-e752c3e0c34e None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:14.018655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07b3e9-e78a-4569-a180-e752c3e0c34e None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:14.020248 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07b3e9-e78a-4569-a180-e752c3e0c34e None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:14.023715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 168/348] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:13 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:14.024702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07b3e9-e78a-4569-a180-e752c3e0c34e None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:14.063083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a01a61-ed15-407a-a405-0a91d53acb6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:14.066091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a01a61-ed15-407a-a405-0a91d53acb6d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:14.145169 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:14.171729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07b3e9-e78a-4569-a180-e752c3e0c34e None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:14.211402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 181/349] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:14.213468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9755bc-3ada-4ae4-a343-941298341786 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:14.213761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9755bc-3ada-4ae4-a343-941298341786 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:14.214036 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9755bc-3ada-4ae4-a343-941298341786 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:14.215581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 182/350] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:14.216351 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9755bc-3ada-4ae4-a343-941298341786 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:14.226688 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0138a5-5312-421c-8632-67cbe93921c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:14.227416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0138a5-5312-421c-8632-67cbe93921c4 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:14.227707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0138a5-5312-421c-8632-67cbe93921c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:14.227988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0138a5-5312-421c-8632-67cbe93921c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:14.242816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:14.322724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0138a5-5312-421c-8632-67cbe93921c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:14.324491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0138a5-5312-421c-8632-67cbe93921c4 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:14.358126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9755bc-3ada-4ae4-a343-941298341786 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:14.410265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dff5d7-2064-4fc8-a339-fef8b1335c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:14.411244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dff5d7-2064-4fc8-a339-fef8b1335c94 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:14.411569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dff5d7-2064-4fc8-a339-fef8b1335c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:14.411852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dff5d7-2064-4fc8-a339-fef8b1335c94 None admin] PATH_INFO: `/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:14.424136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dff5d7-2064-4fc8-a339-fef8b1335c94 None admin] RBAC: Authorizing `identity:get_group(group_id=1b2ffec605c64ebda5b1c4f470c133a3)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:14.426249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dff5d7-2064-4fc8-a339-fef8b1335c94 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:14.427782 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 183/351] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jun 15 10:02:14 2022] GET /identity/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:14.428920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147a77e-bcd7-4e7f-9d8e-22405906f607 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:14.429314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147a77e-bcd7-4e7f-9d8e-22405906f607 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:14.429604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147a77e-bcd7-4e7f-9d8e-22405906f607 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:14.431121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 184/352] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:14.431841 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147a77e-bcd7-4e7f-9d8e-22405906f607 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:14.460063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 169/353] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:14.462387 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d910a-3665-45d4-9ff6-44b1643d53db None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:14.462677 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d910a-3665-45d4-9ff6-44b1643d53db None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:14.462960 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d910a-3665-45d4-9ff6-44b1643d53db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:14.483651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:14.490240 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-8c7d910a-3665-45d4-9ff6-44b1643d53db None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:14.490460 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-8c7d910a-3665-45d4-9ff6-44b1643d53db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:14.534680 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:14.578475 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147a77e-bcd7-4e7f-9d8e-22405906f607 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:14.579134 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 170/354] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:14.580015 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d910a-3665-45d4-9ff6-44b1643d53db None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:14.620644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8023399-4b6e-4bb1-89cd-a577a16c83cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:14.621580 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8023399-4b6e-4bb1-89cd-a577a16c83cc None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:14.621864 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8023399-4b6e-4bb1-89cd-a577a16c83cc None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:14.622302 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8023399-4b6e-4bb1-89cd-a577a16c83cc None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:14.623506 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8023399-4b6e-4bb1-89cd-a577a16c83cc None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:14.625219 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8023399-4b6e-4bb1-89cd-a577a16c83cc None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:14.627036 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 185/355] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:14 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:14.628388 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea9f21-05b9-4570-b43a-8517243c0a4b None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:14.628673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea9f21-05b9-4570-b43a-8517243c0a4b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:14.629354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea9f21-05b9-4570-b43a-8517243c0a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:14.647269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-46ea9f21-05b9-4570-b43a-8517243c0a4b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:14.648435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-46ea9f21-05b9-4570-b43a-8517243c0a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:14.698564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d910a-3665-45d4-9ff6-44b1643d53db None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:14.731896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 186/356] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:14.733259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea9f21-05b9-4570-b43a-8517243c0a4b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:14.742018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07d487-3ed3-4304-9b22-691e97737e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:14.742326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07d487-3ed3-4304-9b22-691e97737e14 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:14.742563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07d487-3ed3-4304-9b22-691e97737e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:14.742839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07d487-3ed3-4304-9b22-691e97737e14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:14.743693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07d487-3ed3-4304-9b22-691e97737e14 None admin] RBAC: Adding query filter params (group.id=1b2ffec605c64ebda5b1c4f470c133a3, role.id=d241ebe8f5654f46a5a92be6f6b0cc36, scope.project.id=2710654d490f43b0adadfd7a62df84bd) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:14.744012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07d487-3ed3-4304-9b22-691e97737e14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:14.745896 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07d487-3ed3-4304-9b22-691e97737e14 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:14.751935 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 171/357] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jun 15 10:02:14 2022] GET /identity/v3/role_assignments?group.id=1b2ffec605c64ebda5b1c4f470c133a3&scope.project.id=2710654d490f43b0adadfd7a62df84bd&role.id=d241ebe8f5654f46a5a92be6f6b0cc36 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:14.753421 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2433d90-4ba4-4233-b5c2-82385ce183cd None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:14.753876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2433d90-4ba4-4233-b5c2-82385ce183cd None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:14.754296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2433d90-4ba4-4233-b5c2-82385ce183cd None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:14.756087 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 172/358] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:14.757324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2433d90-4ba4-4233-b5c2-82385ce183cd None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:14.779843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:14.818480 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:14.894139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea9f21-05b9-4570-b43a-8517243c0a4b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:14.916623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2433d90-4ba4-4233-b5c2-82385ce183cd None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:14.937063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41031b-593d-49d0-a6f2-667aa94c506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:14.938285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41031b-593d-49d0-a6f2-667aa94c506e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:14.938706 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41031b-593d-49d0-a6f2-667aa94c506e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:14.939165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41031b-593d-49d0-a6f2-667aa94c506e None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:14.940544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41031b-593d-49d0-a6f2-667aa94c506e None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:14.941907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41031b-593d-49d0-a6f2-667aa94c506e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:14.943830 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 187/359] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:14 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:14.944828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41031b-593d-49d0-a6f2-667aa94c506e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:14.960187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67c0ef-72d9-4962-9f89-8035db1d5a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:14.961210 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67c0ef-72d9-4962-9f89-8035db1d5a85 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:14.961497 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67c0ef-72d9-4962-9f89-8035db1d5a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:14.961765 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67c0ef-72d9-4962-9f89-8035db1d5a85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:14.976652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67c0ef-72d9-4962-9f89-8035db1d5a85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-ce67c0ef-72d9-4962-9f89-8035db1d5a85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:14.985740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:14.986637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:14.987502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:14.987502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:14.987502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:14.987502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:14.987502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:14.987502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:14.987502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:14.987502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:14.987502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:14.987502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:14.987502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:14.987502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:14.987502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 173/360] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:02:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:14.988647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67c0ef-72d9-4962-9f89-8035db1d5a85 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:15.062197 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:15.079061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41031b-593d-49d0-a6f2-667aa94c506e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:15.115331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:15.124019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67c0ef-72d9-4962-9f89-8035db1d5a85 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:15.128394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845e088-14df-4a10-92ca-02fbe83c4014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:15.129310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845e088-14df-4a10-92ca-02fbe83c4014 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:15.129594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845e088-14df-4a10-92ca-02fbe83c4014 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:15.136105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845e088-14df-4a10-92ca-02fbe83c4014 None admin] PATH_INFO: `/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:15.136105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845e088-14df-4a10-92ca-02fbe83c4014 None admin] RBAC: Authorizing `identity:get_group(group_id=1b2ffec605c64ebda5b1c4f470c133a3)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:15.136105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845e088-14df-4a10-92ca-02fbe83c4014 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:15.136105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 188/361] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jun 15 10:02:14 2022] GET /identity/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:15.137941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab3e27-a382-4ea4-b2a2-7e2c4c925cbb None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:15.138485 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab3e27-a382-4ea4-b2a2-7e2c4c925cbb None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:15.139024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab3e27-a382-4ea4-b2a2-7e2c4c925cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:15.162849 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-b8ab3e27-a382-4ea4-b2a2-7e2c4c925cbb None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:15.164028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b8ab3e27-a382-4ea4-b2a2-7e2c4c925cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:15.166483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c68d6c-94ca-459f-95c8-64825a1d8e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:15.167393 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c68d6c-94ca-459f-95c8-64825a1d8e46 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:15.167683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c68d6c-94ca-459f-95c8-64825a1d8e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:15.168100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c68d6c-94ca-459f-95c8-64825a1d8e46 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:15.181190 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c68d6c-94ca-459f-95c8-64825a1d8e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-e3c68d6c-94ca-459f-95c8-64825a1d8e46 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:15.190320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:15.191247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:15.192018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:15.192018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:15.192018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:15.192018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:15.192018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:15.192018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:15.192018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:15.192018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:15.192018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:15.192018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:15.192018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 15 10:02:15.192018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:15.194892 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 174/362] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:15.195732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c68d6c-94ca-459f-95c8-64825a1d8e46 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:15.268985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 189/363] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:15.268985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab3e27-a382-4ea4-b2a2-7e2c4c925cbb None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:15.335723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c68d6c-94ca-459f-95c8-64825a1d8e46 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:15.372241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:15.384564 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab3e27-a382-4ea4-b2a2-7e2c4c925cbb None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:15.385417 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e7198-807b-405e-af57-a2e50a8aac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:15.386295 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e7198-807b-405e-af57-a2e50a8aac57 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:15.386582 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e7198-807b-405e-af57-a2e50a8aac57 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:15.386867 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e7198-807b-405e-af57-a2e50a8aac57 None admin] PATH_INFO: `/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:15.388092 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e7198-807b-405e-af57-a2e50a8aac57 None admin] RBAC: Authorizing `identity:get_group(group_id=1b2ffec605c64ebda5b1c4f470c133a3)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:15.389605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e7198-807b-405e-af57-a2e50a8aac57 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:15.391826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 175/364] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jun 15 10:02:15 2022] GET /identity/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:15.392616 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e7198-807b-405e-af57-a2e50a8aac57 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:15.425481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:15.426143 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604aa00b-9f7b-4f76-a7e5-a3da7bfaa1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:15.427129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604aa00b-9f7b-4f76-a7e5-a3da7bfaa1fc None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:15.427622 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604aa00b-9f7b-4f76-a7e5-a3da7bfaa1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:15.428344 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604aa00b-9f7b-4f76-a7e5-a3da7bfaa1fc None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:15.429968 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604aa00b-9f7b-4f76-a7e5-a3da7bfaa1fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:15.430531 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604aa00b-9f7b-4f76-a7e5-a3da7bfaa1fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:15.432166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604aa00b-9f7b-4f76-a7e5-a3da7bfaa1fc None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:15.445683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 190/365] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:15.446394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604aa00b-9f7b-4f76-a7e5-a3da7bfaa1fc None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:15.544404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e7198-807b-405e-af57-a2e50a8aac57 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:15.591180 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e727e9f-2eb1-4dc7-8549-1b4801194678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:15.591860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e727e9f-2eb1-4dc7-8549-1b4801194678 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:15.592150 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e727e9f-2eb1-4dc7-8549-1b4801194678 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:15.592419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e727e9f-2eb1-4dc7-8549-1b4801194678 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:15.593310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e727e9f-2eb1-4dc7-8549-1b4801194678 None admin] RBAC: Adding query filter params (group.id=1b2ffec605c64ebda5b1c4f470c133a3, role.id=c91413ec3feb454ca7186eb52a57ed96, scope.project.id=2710654d490f43b0adadfd7a62df84bd) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:15.594249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e727e9f-2eb1-4dc7-8549-1b4801194678 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:15.595715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e727e9f-2eb1-4dc7-8549-1b4801194678 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:15.603413 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 176/366] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jun 15 10:02:15 2022] GET /identity/v3/role_assignments?group.id=1b2ffec605c64ebda5b1c4f470c133a3&scope.project.id=2710654d490f43b0adadfd7a62df84bd&role.id=c91413ec3feb454ca7186eb52a57ed96 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:15.604093 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e727e9f-2eb1-4dc7-8549-1b4801194678 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:15.604474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604aa00b-9f7b-4f76-a7e5-a3da7bfaa1fc None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:15.668139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbf743-ebbb-4d2f-9911-3aa7b011711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:15.668932 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbf743-ebbb-4d2f-9911-3aa7b011711f None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:15.669256 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbf743-ebbb-4d2f-9911-3aa7b011711f None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:15.669536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbf743-ebbb-4d2f-9911-3aa7b011711f None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:15.670662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbf743-ebbb-4d2f-9911-3aa7b011711f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:15.670982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbf743-ebbb-4d2f-9911-3aa7b011711f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:15.672923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbf743-ebbb-4d2f-9911-3aa7b011711f None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:15.685284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 191/367] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:02:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:15.686114 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbf743-ebbb-4d2f-9911-3aa7b011711f None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:15.711067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:15.738008 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:15.754924 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e727e9f-2eb1-4dc7-8549-1b4801194678 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:15.808813 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c553f-9fe5-4387-8c4b-63ade3a5c3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:15.809652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c553f-9fe5-4387-8c4b-63ade3a5c3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:15.809932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c553f-9fe5-4387-8c4b-63ade3a5c3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:15.810209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c553f-9fe5-4387-8c4b-63ade3a5c3c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:15.824353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c553f-9fe5-4387-8c4b-63ade3a5c3c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:15.831995 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbf743-ebbb-4d2f-9911-3aa7b011711f None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-2a3c553f-9fe5-4387-8c4b-63ade3a5c3c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:15.833054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:15.833952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:15.834799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:15.834799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:15.834799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:15.834799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:15.834799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:15.834799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:15.834799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:15.834799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:15.834799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:15.834799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:15.834799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:02:15.834799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:15.835430 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 177/368] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:02:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:15.836010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c553f-9fe5-4387-8c4b-63ade3a5c3c8 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:15.874200 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d39242-428b-4d69-87ce-eb1343fef11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:15.874964 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d39242-428b-4d69-87ce-eb1343fef11e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:15.875251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d39242-428b-4d69-87ce-eb1343fef11e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:15.875532 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d39242-428b-4d69-87ce-eb1343fef11e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:15.876442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d39242-428b-4d69-87ce-eb1343fef11e None admin] RBAC: Adding query filter params (group.id=1b2ffec605c64ebda5b1c4f470c133a3, role.id=c91413ec3feb454ca7186eb52a57ed96, scope.project.id=354b4a7dec454bbe9294b154303e7f7e) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:15.876820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d39242-428b-4d69-87ce-eb1343fef11e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:15.878317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d39242-428b-4d69-87ce-eb1343fef11e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:15.884729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 192/369] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jun 15 10:02:15 2022] GET /identity/v3/role_assignments?group.id=1b2ffec605c64ebda5b1c4f470c133a3&scope.project.id=354b4a7dec454bbe9294b154303e7f7e&role.id=c91413ec3feb454ca7186eb52a57ed96 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:15.885483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d39242-428b-4d69-87ce-eb1343fef11e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:15.961262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c553f-9fe5-4387-8c4b-63ade3a5c3c8 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:15.975346 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:16.003555 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d39242-428b-4d69-87ce-eb1343fef11e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:16.005692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294cacb8-2d77-4559-b5d5-47bf72bbbc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:16.006369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cacb8-2d77-4559-b5d5-47bf72bbbc4b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:16.006658 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cacb8-2d77-4559-b5d5-47bf72bbbc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:16.006932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cacb8-2d77-4559-b5d5-47bf72bbbc4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:16.033505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:16.045633 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c11e8-4806-4181-83af-98e3eb5dc87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:16.046578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c11e8-4806-4181-83af-98e3eb5dc87f None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:16.047121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c11e8-4806-4181-83af-98e3eb5dc87f None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:16.047444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c11e8-4806-4181-83af-98e3eb5dc87f None admin] PATH_INFO: `/v3/users/0a61db65f4cd4d2094c9e304cc3d9766` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:16.049199 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c11e8-4806-4181-83af-98e3eb5dc87f None admin] RBAC: Authorizing `identity:get_user(user_id=0a61db65f4cd4d2094c9e304cc3d9766)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:16.050697 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c11e8-4806-4181-83af-98e3eb5dc87f None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:16.053214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 193/370] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:02:15 2022] GET /identity/v3/users/0a61db65f4cd4d2094c9e304cc3d9766 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:16.054009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c11e8-4806-4181-83af-98e3eb5dc87f None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:16.103069 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294cacb8-2d77-4559-b5d5-47bf72bbbc4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:16.104641 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294cacb8-2d77-4559-b5d5-47bf72bbbc4b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:16.210092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c11e8-4806-4181-83af-98e3eb5dc87f None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:16.227234 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 178/371] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:16.228190 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294cacb8-2d77-4559-b5d5-47bf72bbbc4b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:16.253569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aafc2e-0ea1-4304-8c37-f38b93a073b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:16.254270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aafc2e-0ea1-4304-8c37-f38b93a073b5 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:16.254556 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aafc2e-0ea1-4304-8c37-f38b93a073b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:16.254835 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aafc2e-0ea1-4304-8c37-f38b93a073b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:16.255601 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aafc2e-0ea1-4304-8c37-f38b93a073b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:16.255913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aafc2e-0ea1-4304-8c37-f38b93a073b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:16.257351 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aafc2e-0ea1-4304-8c37-f38b93a073b5 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:16.266646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 194/372] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:16.271536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aafc2e-0ea1-4304-8c37-f38b93a073b5 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:16.284205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:16.329350 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:16.343867 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294cacb8-2d77-4559-b5d5-47bf72bbbc4b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:16.391831 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd0033-3b3d-4daf-bc43-32eaecf6e274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:16.392494 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd0033-3b3d-4daf-bc43-32eaecf6e274 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:16.392775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd0033-3b3d-4daf-bc43-32eaecf6e274 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:16.393070 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd0033-3b3d-4daf-bc43-32eaecf6e274 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:16.393918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd0033-3b3d-4daf-bc43-32eaecf6e274 None admin] RBAC: Adding query filter params (role.id=127c304ac7ee4fce8de777dbd4037b59, user.id=0a61db65f4cd4d2094c9e304cc3d9766, scope.system=all) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:16.394231 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd0033-3b3d-4daf-bc43-32eaecf6e274 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:16.395881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd0033-3b3d-4daf-bc43-32eaecf6e274 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:16.415096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 179/373] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Jun 15 10:02:16 2022] GET /identity/v3/role_assignments?user.id=0a61db65f4cd4d2094c9e304cc3d9766&scope.system=all&role.id=127c304ac7ee4fce8de777dbd4037b59 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:16.416061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd0033-3b3d-4daf-bc43-32eaecf6e274 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:16.486650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aafc2e-0ea1-4304-8c37-f38b93a073b5 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:16.565700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f14743-ec51-43ed-b5d0-940621588701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:16.566593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f14743-ec51-43ed-b5d0-940621588701 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:16.566874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f14743-ec51-43ed-b5d0-940621588701 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:16.567149 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f14743-ec51-43ed-b5d0-940621588701 None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:16.573950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f14743-ec51-43ed-b5d0-940621588701 None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:16.575456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f14743-ec51-43ed-b5d0-940621588701 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:16.577672 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 195/374] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:16 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:16.581439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd0033-3b3d-4daf-bc43-32eaecf6e274 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:16.583149 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f14743-ec51-43ed-b5d0-940621588701 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:16.594058 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:16.636698 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa1209-2783-49da-b22b-e06ff2b652fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:16.638106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa1209-2783-49da-b22b-e06ff2b652fc None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:16.638406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa1209-2783-49da-b22b-e06ff2b652fc None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:16.638687 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa1209-2783-49da-b22b-e06ff2b652fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:16.750390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa1209-2783-49da-b22b-e06ff2b652fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:16.751789 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa1209-2783-49da-b22b-e06ff2b652fc None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:16.823905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:16.876525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f14743-ec51-43ed-b5d0-940621588701 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:16.936998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d85f1-b44f-4b8d-bb64-ce2d6a55e0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:16.938423 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d85f1-b44f-4b8d-bb64-ce2d6a55e0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:16.938707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d85f1-b44f-4b8d-bb64-ce2d6a55e0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:16.938973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d85f1-b44f-4b8d-bb64-ce2d6a55e0c8 None admin] PATH_INFO: `/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:16.940423 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d85f1-b44f-4b8d-bb64-ce2d6a55e0c8 None admin] RBAC: Authorizing `identity:get_group(group_id=1b2ffec605c64ebda5b1c4f470c133a3)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:16.941903 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d85f1-b44f-4b8d-bb64-ce2d6a55e0c8 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:16.945399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 196/375] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jun 15 10:02:16 2022] GET /identity/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3 => generated 226 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:16.949571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d85f1-b44f-4b8d-bb64-ce2d6a55e0c8 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:16.967888 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 180/376] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:16.976927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa1209-2783-49da-b22b-e06ff2b652fc None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:16.987864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:17.119552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d85f1-b44f-4b8d-bb64-ce2d6a55e0c8 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:17.169373 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca213da2-41ff-4135-a264-70a5a0ad583b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:17.170185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca213da2-41ff-4135-a264-70a5a0ad583b None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:17.170466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca213da2-41ff-4135-a264-70a5a0ad583b None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:17.170741 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca213da2-41ff-4135-a264-70a5a0ad583b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:17.172114 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca213da2-41ff-4135-a264-70a5a0ad583b None admin] RBAC: Adding query filter params (group.id=1b2ffec605c64ebda5b1c4f470c133a3, role.id=d241ebe8f5654f46a5a92be6f6b0cc36, scope.project.id=354b4a7dec454bbe9294b154303e7f7e) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:17.172553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca213da2-41ff-4135-a264-70a5a0ad583b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:17.174522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca213da2-41ff-4135-a264-70a5a0ad583b None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:17.181478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 197/377] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jun 15 10:02:16 2022] GET /identity/v3/role_assignments?group.id=1b2ffec605c64ebda5b1c4f470c133a3&scope.project.id=354b4a7dec454bbe9294b154303e7f7e&role.id=d241ebe8f5654f46a5a92be6f6b0cc36 => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:17.228831 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa1209-2783-49da-b22b-e06ff2b652fc None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:17.277239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9f2b7-ea07-4625-bd4b-21b12309a10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:17.278133 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9f2b7-ea07-4625-bd4b-21b12309a10f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:17.278417 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9f2b7-ea07-4625-bd4b-21b12309a10f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:17.278692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9f2b7-ea07-4625-bd4b-21b12309a10f None admin] PATH_INFO: `/v3/projects/8232a31392e948a78496aa9a0a9a81a0` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:17.280428 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9f2b7-ea07-4625-bd4b-21b12309a10f None admin] RBAC: Authorizing `identity:get_project(project_id=8232a31392e948a78496aa9a0a9a81a0)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:17.282825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9f2b7-ea07-4625-bd4b-21b12309a10f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:17.284922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 181/378] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:16 2022] GET /identity/v3/projects/8232a31392e948a78496aa9a0a9a81a0 => generated 344 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:17.290357 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca213da2-41ff-4135-a264-70a5a0ad583b None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:17.313346 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:17.451074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca213da2-41ff-4135-a264-70a5a0ad583b None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:17.491319 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec2943-c3a2-4216-8479-d0469f9c3947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:17.492278 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec2943-c3a2-4216-8479-d0469f9c3947 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:17.492567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec2943-c3a2-4216-8479-d0469f9c3947 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:17.492845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec2943-c3a2-4216-8479-d0469f9c3947 None admin] PATH_INFO: `/v3/groups/0d0b8234732741a4afb8034bd50a07fc` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:17.498287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec2943-c3a2-4216-8479-d0469f9c3947 None admin] RBAC: Authorizing `identity:get_group(group_id=0d0b8234732741a4afb8034bd50a07fc)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:17.500252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec2943-c3a2-4216-8479-d0469f9c3947 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:17.507701 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 198/379] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jun 15 10:02:17 2022] GET /identity/v3/groups/0d0b8234732741a4afb8034bd50a07fc => generated 229 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:17.508653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9f2b7-ea07-4625-bd4b-21b12309a10f None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:17.624439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:17.668900 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9f2b7-ea07-4625-bd4b-21b12309a10f None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:17.726594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773bc10-627f-494b-ad79-6c3e3973f0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:17.727287 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773bc10-627f-494b-ad79-6c3e3973f0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:17.727570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773bc10-627f-494b-ad79-6c3e3973f0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:17.727845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773bc10-627f-494b-ad79-6c3e3973f0e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:17.728708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773bc10-627f-494b-ad79-6c3e3973f0e5 None admin] RBAC: Adding query filter params (group.id=0d0b8234732741a4afb8034bd50a07fc, role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=8232a31392e948a78496aa9a0a9a81a0) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:17.729049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773bc10-627f-494b-ad79-6c3e3973f0e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:17.730527 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773bc10-627f-494b-ad79-6c3e3973f0e5 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:17.737985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 182/380] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jun 15 10:02:17 2022] GET /identity/v3/role_assignments?group.id=0d0b8234732741a4afb8034bd50a07fc&scope.project.id=8232a31392e948a78496aa9a0a9a81a0&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:17.787249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422794ea-7ee9-489d-ac70-311dc5cf5462 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:17.787584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422794ea-7ee9-489d-ac70-311dc5cf5462 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:17.787819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422794ea-7ee9-489d-ac70-311dc5cf5462 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:17.789324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 199/381] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:17.795412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679a41f-a35d-4b7f-b628-284b830fb023 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:17.795707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679a41f-a35d-4b7f-b628-284b830fb023 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:17.796154 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679a41f-a35d-4b7f-b628-284b830fb023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:17.815479 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-1679a41f-a35d-4b7f-b628-284b830fb023 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:17.816932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1679a41f-a35d-4b7f-b628-284b830fb023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:17.899850 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 183/382] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:17.937887 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422794ea-7ee9-489d-ac70-311dc5cf5462 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:18.034358 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:18.071408 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422794ea-7ee9-489d-ac70-311dc5cf5462 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:18.110244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f857fd-dc3c-40ef-a26b-ba8fafeddb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:18.111084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f857fd-dc3c-40ef-a26b-ba8fafeddb82 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:18.111586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f857fd-dc3c-40ef-a26b-ba8fafeddb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:18.111868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f857fd-dc3c-40ef-a26b-ba8fafeddb82 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:18.125334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f857fd-dc3c-40ef-a26b-ba8fafeddb82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-92f857fd-dc3c-40ef-a26b-ba8fafeddb82 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:18.134973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:18.135894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:18.136677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:18.136677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:18.136677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:18.136677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:18.136677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:18.136677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:18.136677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:18.136677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:18.136677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:18.136677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:18.136677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:18.136677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:18.136677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 200/383] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:02:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:18.143426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679a41f-a35d-4b7f-b628-284b830fb023 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:18.234476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:18.276276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679a41f-a35d-4b7f-b628-284b830fb023 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:18.330723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a92e5-cb56-4416-9bb7-c1b0fc0c5e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:18.331335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a92e5-cb56-4416-9bb7-c1b0fc0c5e7c None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:18.331618 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a92e5-cb56-4416-9bb7-c1b0fc0c5e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:18.331896 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a92e5-cb56-4416-9bb7-c1b0fc0c5e7c None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:18.332683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a92e5-cb56-4416-9bb7-c1b0fc0c5e7c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:18.333084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a92e5-cb56-4416-9bb7-c1b0fc0c5e7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:18.334487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a92e5-cb56-4416-9bb7-c1b0fc0c5e7c None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:18.348026 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 184/384] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:18.353428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f857fd-dc3c-40ef-a26b-ba8fafeddb82 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:18.492714 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f857fd-dc3c-40ef-a26b-ba8fafeddb82 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:18.539870 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:18.546391 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c3c2f-75c6-4463-89bb-68ed7e6f5482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:18.547705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c3c2f-75c6-4463-89bb-68ed7e6f5482 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:18.548157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c3c2f-75c6-4463-89bb-68ed7e6f5482 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:18.548586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c3c2f-75c6-4463-89bb-68ed7e6f5482 None admin] PATH_INFO: `/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:18.550125 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c3c2f-75c6-4463-89bb-68ed7e6f5482 None admin] RBAC: Authorizing `identity:get_group(group_id=1b2ffec605c64ebda5b1c4f470c133a3)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:18.555159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c3c2f-75c6-4463-89bb-68ed7e6f5482 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:18.555159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 201/385] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jun 15 10:02:18 2022] GET /identity/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:18.561581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a92e5-cb56-4416-9bb7-c1b0fc0c5e7c None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:18.596858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e8b27-d9b9-40bb-9b6f-e5b54174960b None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:18.597070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e8b27-d9b9-40bb-9b6f-e5b54174960b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:18.597361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e8b27-d9b9-40bb-9b6f-e5b54174960b None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:18.599608 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 202/386] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:02:18.604995 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ad6d9-d116-4901-afb2-f15f320dccc4 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:18.605281 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ad6d9-d116-4901-afb2-f15f320dccc4 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:18.605561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ad6d9-d116-4901-afb2-f15f320dccc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:18.629539 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-298ad6d9-d116-4901-afb2-f15f320dccc4 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:18.630799 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-298ad6d9-d116-4901-afb2-f15f320dccc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:18.727119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 203/387] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:18.729455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6145e04-5690-4714-8a9b-7f8f6d4ed942 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:18.730064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6145e04-5690-4714-8a9b-7f8f6d4ed942 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:18.730786 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6145e04-5690-4714-8a9b-7f8f6d4ed942 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:18.733408 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a92e5-cb56-4416-9bb7-c1b0fc0c5e7c None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:18.737073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 204/388] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:18.739394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52ea7e-2a75-45c3-858c-2b0d817a8829 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:18.741361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52ea7e-2a75-45c3-858c-2b0d817a8829 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:18.741823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52ea7e-2a75-45c3-858c-2b0d817a8829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:18.765801 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-5e52ea7e-2a75-45c3-858c-2b0d817a8829 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:18.768224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5e52ea7e-2a75-45c3-858c-2b0d817a8829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:18.770642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:18.778481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83115c2b-111f-4fa0-bb3b-bc6a36410e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:18.779701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83115c2b-111f-4fa0-bb3b-bc6a36410e4f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:18.780141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83115c2b-111f-4fa0-bb3b-bc6a36410e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:18.780607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83115c2b-111f-4fa0-bb3b-bc6a36410e4f None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:18.785063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83115c2b-111f-4fa0-bb3b-bc6a36410e4f None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:18.785063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83115c2b-111f-4fa0-bb3b-bc6a36410e4f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:18.787046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 185/389] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:18 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:18.788111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83115c2b-111f-4fa0-bb3b-bc6a36410e4f None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:18.851921 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 205/390] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:18.853079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52ea7e-2a75-45c3-858c-2b0d817a8829 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:18.903206 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83115c2b-111f-4fa0-bb3b-bc6a36410e4f None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:18.929551 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:18.942503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc9dd3-0d0b-4c0d-ac66-7ae44a612176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:18.943573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc9dd3-0d0b-4c0d-ac66-7ae44a612176 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:18.944021 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc9dd3-0d0b-4c0d-ac66-7ae44a612176 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:18.944441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc9dd3-0d0b-4c0d-ac66-7ae44a612176 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:18.956694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc9dd3-0d0b-4c0d-ac66-7ae44a612176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-fafc9dd3-0d0b-4c0d-ac66-7ae44a612176 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:18.964878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:18.965982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:18.966909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:18.966909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:18.966909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:18.966909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:18.966909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:18.966909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:18.966909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:18.966909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:18.966909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:18.966909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:18.966909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:18.966909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:18.970380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 186/391] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:18.971501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc9dd3-0d0b-4c0d-ac66-7ae44a612176 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:18.975217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52ea7e-2a75-45c3-858c-2b0d817a8829 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:19.013174 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6196926-90b7-4b42-b3f8-f10fc5c8791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:19.014119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6196926-90b7-4b42-b3f8-f10fc5c8791d None admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:19.014402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6196926-90b7-4b42-b3f8-f10fc5c8791d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:19.014683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6196926-90b7-4b42-b3f8-f10fc5c8791d None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd/groups/1b2ffec605c64ebda5b1c4f470c133a3/roles/d241ebe8f5654f46a5a92be6f6b0cc36` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:19.017665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6196926-90b7-4b42-b3f8-f10fc5c8791d None admin] RBAC: Authorizing `identity:create_grant(project_id=2710654d490f43b0adadfd7a62df84bd, group_id=1b2ffec605c64ebda5b1c4f470c133a3, role_id=d241ebe8f5654f46a5a92be6f6b0cc36)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:19.019199 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6196926-90b7-4b42-b3f8-f10fc5c8791d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:19.037888 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:19.079910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc9dd3-0d0b-4c0d-ac66-7ae44a612176 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:19.098792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 206/392] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Jun 15 10:02:18 2022] PUT /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd/groups/1b2ffec605c64ebda5b1c4f470c133a3/roles/d241ebe8f5654f46a5a92be6f6b0cc36 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:02:19.100368 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6196926-90b7-4b42-b3f8-f10fc5c8791d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:19.155236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c0075-1a43-46de-9176-e37c7b717e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:19.157996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c0075-1a43-46de-9176-e37c7b717e56 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:19.158576 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c0075-1a43-46de-9176-e37c7b717e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:19.159420 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83c0075-1a43-46de-9176-e37c7b717e56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:19.183767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83c0075-1a43-46de-9176-e37c7b717e56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-e83c0075-1a43-46de-9176-e37c7b717e56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:19.201098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:19.202059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:19.203075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:19.203075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:19.203075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:19.203075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:19.203075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:19.203075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:19.203075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:19.203075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:19.203075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:19.203075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:19.203075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:19.203075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:19.208760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 187/393] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:19.215220 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c0075-1a43-46de-9176-e37c7b717e56 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:19.323389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6196926-90b7-4b42-b3f8-f10fc5c8791d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:19.344993 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83c0075-1a43-46de-9176-e37c7b717e56 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:19.358798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:19.367084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:19.378736 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ef036-51be-4b3a-bb81-0fa2bc36bb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:19.379428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ef036-51be-4b3a-bb81-0fa2bc36bb75 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:19.382583 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ef036-51be-4b3a-bb81-0fa2bc36bb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:19.382583 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ef036-51be-4b3a-bb81-0fa2bc36bb75 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:19.382583 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ef036-51be-4b3a-bb81-0fa2bc36bb75 None admin] RBAC: Adding query filter params (name=member) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:19.383022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ef036-51be-4b3a-bb81-0fa2bc36bb75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:19.384498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ef036-51be-4b3a-bb81-0fa2bc36bb75 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:19.390043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e57910-f22b-4440-aa7d-233c991874a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:19.392193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e57910-f22b-4440-aa7d-233c991874a2 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:19.392982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e57910-f22b-4440-aa7d-233c991874a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:19.393442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e57910-f22b-4440-aa7d-233c991874a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:19.394405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e57910-f22b-4440-aa7d-233c991874a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:19.394712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 207/394] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:02:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:19.395238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e57910-f22b-4440-aa7d-233c991874a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:19.397143 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e57910-f22b-4440-aa7d-233c991874a2 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:19.400279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ef036-51be-4b3a-bb81-0fa2bc36bb75 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:19.414038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 188/395] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:02:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:19.422293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e57910-f22b-4440-aa7d-233c991874a2 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:19.611874 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e57910-f22b-4440-aa7d-233c991874a2 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:19.619237 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ef036-51be-4b3a-bb81-0fa2bc36bb75 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:19.661384 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b923be8-d9d4-482a-9243-ad33039e63ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:19.662545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b923be8-d9d4-482a-9243-ad33039e63ca None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:19.662837 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b923be8-d9d4-482a-9243-ad33039e63ca None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:19.663117 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b923be8-d9d4-482a-9243-ad33039e63ca None admin] PATH_INFO: `/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:19.664619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b923be8-d9d4-482a-9243-ad33039e63ca None admin] RBAC: Authorizing `identity:get_group(group_id=1b2ffec605c64ebda5b1c4f470c133a3)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:19.666177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b923be8-d9d4-482a-9243-ad33039e63ca None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:19.668810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 208/396] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jun 15 10:02:19 2022] GET /identity/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3 => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:19.676295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b923be8-d9d4-482a-9243-ad33039e63ca None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:19.684908 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe000da-6a5a-4a93-ae97-dc42434976e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:19.685889 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe000da-6a5a-4a93-ae97-dc42434976e7 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:19.687138 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe000da-6a5a-4a93-ae97-dc42434976e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:19.687510 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe000da-6a5a-4a93-ae97-dc42434976e7 None admin] PATH_INFO: `/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:19.688789 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe000da-6a5a-4a93-ae97-dc42434976e7 None admin] RBAC: Authorizing `identity:get_group(group_id=1b2ffec605c64ebda5b1c4f470c133a3)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:19.695444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe000da-6a5a-4a93-ae97-dc42434976e7 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:19.704374 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 189/397] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jun 15 10:02:19 2022] GET /identity/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3 => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:19.705350 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe000da-6a5a-4a93-ae97-dc42434976e7 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:19.779999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:19.795004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:19.865302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe000da-6a5a-4a93-ae97-dc42434976e7 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:19.905712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af350d93-7f74-4dab-8702-efaf11088d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:19.907288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af350d93-7f74-4dab-8702-efaf11088d2e None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:19.907569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af350d93-7f74-4dab-8702-efaf11088d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:19.907842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af350d93-7f74-4dab-8702-efaf11088d2e None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:19.909963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af350d93-7f74-4dab-8702-efaf11088d2e None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:19.912182 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af350d93-7f74-4dab-8702-efaf11088d2e None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:19.914497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b923be8-d9d4-482a-9243-ad33039e63ca None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:19.915023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 190/398] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:19 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:19.916819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8c120-a168-49e8-8f4a-efda7408ba46 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:19.917354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8c120-a168-49e8-8f4a-efda7408ba46 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:19.917846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8c120-a168-49e8-8f4a-efda7408ba46 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:19.919435 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 191/399] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:19.920418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8c120-a168-49e8-8f4a-efda7408ba46 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:19.965215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c2d3b-ac7c-4887-9e03-7ea4269e18df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:19.966097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c2d3b-ac7c-4887-9e03-7ea4269e18df None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:19.966376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c2d3b-ac7c-4887-9e03-7ea4269e18df None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:19.966661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c2d3b-ac7c-4887-9e03-7ea4269e18df None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:19.968522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c2d3b-ac7c-4887-9e03-7ea4269e18df None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:19.971137 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c2d3b-ac7c-4887-9e03-7ea4269e18df None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:19.974055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 209/400] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:19 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:19.975547 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c1af8-fb65-4539-98e4-5d081c05bdb6 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:19.976080 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c1af8-fb65-4539-98e4-5d081c05bdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:19.976163 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c1af8-fb65-4539-98e4-5d081c05bdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:19.995700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-be8c1af8-fb65-4539-98e4-5d081c05bdb6 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:19.997359 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-be8c1af8-fb65-4539-98e4-5d081c05bdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:20.053581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8c120-a168-49e8-8f4a-efda7408ba46 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:20.072152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 210/401] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:20.073501 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c1af8-fb65-4539-98e4-5d081c05bdb6 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:20.087063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:20.117339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50eb55-9508-42bf-b466-dcb4e8ba9a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:20.118927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50eb55-9508-42bf-b466-dcb4e8ba9a15 None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:20.119576 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50eb55-9508-42bf-b466-dcb4e8ba9a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:20.120394 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50eb55-9508-42bf-b466-dcb4e8ba9a15 None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd/groups/1b2ffec605c64ebda5b1c4f470c133a3/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:20.123165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50eb55-9508-42bf-b466-dcb4e8ba9a15 None admin] RBAC: Authorizing `identity:create_grant(project_id=2710654d490f43b0adadfd7a62df84bd, group_id=1b2ffec605c64ebda5b1c4f470c133a3, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:20.126174 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50eb55-9508-42bf-b466-dcb4e8ba9a15 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:20.139670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:20.182949 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 192/402] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Jun 15 10:02:19 2022] PUT /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd/groups/1b2ffec605c64ebda5b1c4f470c133a3/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:02:20.184360 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50eb55-9508-42bf-b466-dcb4e8ba9a15 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:20.197443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c1af8-fb65-4539-98e4-5d081c05bdb6 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:20.265187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5ef39-4a9c-4fa5-9ef1-eca1056009c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:20.266734 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5ef39-4a9c-4fa5-9ef1-eca1056009c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:20.267062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5ef39-4a9c-4fa5-9ef1-eca1056009c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:20.267310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5ef39-4a9c-4fa5-9ef1-eca1056009c9 None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e/groups/1b2ffec605c64ebda5b1c4f470c133a3/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:20.278915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5ef39-4a9c-4fa5-9ef1-eca1056009c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=354b4a7dec454bbe9294b154303e7f7e, group_id=1b2ffec605c64ebda5b1c4f470c133a3, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:20.280483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5ef39-4a9c-4fa5-9ef1-eca1056009c9 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:20.322124 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50eb55-9508-42bf-b466-dcb4e8ba9a15 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:20.369848 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b932df9-bfde-4a2c-b0de-2a4cca9dfa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:20.370667 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b932df9-bfde-4a2c-b0de-2a4cca9dfa20 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:20.370950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b932df9-bfde-4a2c-b0de-2a4cca9dfa20 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:20.371225 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b932df9-bfde-4a2c-b0de-2a4cca9dfa20 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:20.387092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 211/403] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Jun 15 10:02:20 2022] PUT /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e/groups/1b2ffec605c64ebda5b1c4f470c133a3/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:02:20.395712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b932df9-bfde-4a2c-b0de-2a4cca9dfa20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-9b932df9-bfde-4a2c-b0de-2a4cca9dfa20 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:20.405035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:20.406543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:20.407572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:20.407572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:20.407572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:20.407572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:20.407572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:20.407572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:20.407572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:20.407572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:20.407572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:20.407572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:20.407572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:20.407572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:20.410495 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 193/404] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:02:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:20.416536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5ef39-4a9c-4fa5-9ef1-eca1056009c9 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:20.555173 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc6210-1261-4aba-8ed8-aff818ef7d8c None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:20.555483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc6210-1261-4aba-8ed8-aff818ef7d8c None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:20.555747 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc6210-1261-4aba-8ed8-aff818ef7d8c None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:20.557485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 194/405] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:20.563907 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8265594-61b9-40ae-b338-79899fefd2bc None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:20.564195 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8265594-61b9-40ae-b338-79899fefd2bc None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:20.564478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8265594-61b9-40ae-b338-79899fefd2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:20.568510 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5ef39-4a9c-4fa5-9ef1-eca1056009c9 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:20.580564 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:20.610728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b8265594-61b9-40ae-b338-79899fefd2bc None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:20.612090 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b8265594-61b9-40ae-b338-79899fefd2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:20.618299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167f89a-9d21-462b-9d3e-a7d139d7586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:20.619363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167f89a-9d21-462b-9d3e-a7d139d7586c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:20.619873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167f89a-9d21-462b-9d3e-a7d139d7586c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:20.620510 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c167f89a-9d21-462b-9d3e-a7d139d7586c None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:20.621470 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167f89a-9d21-462b-9d3e-a7d139d7586c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:20.621935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167f89a-9d21-462b-9d3e-a7d139d7586c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:20.623566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c167f89a-9d21-462b-9d3e-a7d139d7586c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:20.637166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 212/406] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:20.643515 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167f89a-9d21-462b-9d3e-a7d139d7586c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:20.647870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:20.706679 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 195/407] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:02:20.741807 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8265594-61b9-40ae-b338-79899fefd2bc None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:20.797989 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c167f89a-9d21-462b-9d3e-a7d139d7586c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:20.851297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72533db6-dab7-4d56-b43a-9568671758c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:20.852522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72533db6-dab7-4d56-b43a-9568671758c1 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:20.854367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72533db6-dab7-4d56-b43a-9568671758c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:20.854884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72533db6-dab7-4d56-b43a-9568671758c1 None admin] PATH_INFO: `/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:20.857388 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72533db6-dab7-4d56-b43a-9568671758c1 None admin] RBAC: Authorizing `identity:get_group(group_id=1b2ffec605c64ebda5b1c4f470c133a3)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:20.860353 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72533db6-dab7-4d56-b43a-9568671758c1 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:20.867328 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 213/408] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jun 15 10:02:20 2022] GET /identity/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:20.870207 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72533db6-dab7-4d56-b43a-9568671758c1 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:20.923963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8265594-61b9-40ae-b338-79899fefd2bc None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:20.965420 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b210218-ba40-452f-96b7-7f5c712be1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:20.966847 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b210218-ba40-452f-96b7-7f5c712be1aa None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:20.967350 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b210218-ba40-452f-96b7-7f5c712be1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:20.967828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b210218-ba40-452f-96b7-7f5c712be1aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:20.980861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b210218-ba40-452f-96b7-7f5c712be1aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-2b210218-ba40-452f-96b7-7f5c712be1aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:20.990383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:20.991299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:20.992086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:20.992086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:20.992086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:20.992086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:20.992086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:20.992086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:20.992086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:20.992086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:20.992086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:20.992086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:20.992086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:02:20.992086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:20.993591 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:20.995446 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 196/409] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:02:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:21.006260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b210218-ba40-452f-96b7-7f5c712be1aa None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:21.059548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:21.097509 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72533db6-dab7-4d56-b43a-9568671758c1 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:21.168824 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27893414-26d0-4513-b607-f707a9596164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:21.170631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27893414-26d0-4513-b607-f707a9596164 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:21.170913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27893414-26d0-4513-b607-f707a9596164 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:21.171193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27893414-26d0-4513-b607-f707a9596164 None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:21.173715 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27893414-26d0-4513-b607-f707a9596164 None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:21.176223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27893414-26d0-4513-b607-f707a9596164 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:21.179160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 214/410] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:20 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:21.184124 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b210218-ba40-452f-96b7-7f5c712be1aa None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:21.185896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27893414-26d0-4513-b607-f707a9596164 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:21.252019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0b631-1662-439b-b7ee-bfeca28650da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:21.252703 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0b631-1662-439b-b7ee-bfeca28650da None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:21.253074 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0b631-1662-439b-b7ee-bfeca28650da None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:21.253419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0b631-1662-439b-b7ee-bfeca28650da None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:21.254255 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0b631-1662-439b-b7ee-bfeca28650da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:21.254569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0b631-1662-439b-b7ee-bfeca28650da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:21.255984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0b631-1662-439b-b7ee-bfeca28650da None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:21.270600 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 197/411] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:21.275876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0b631-1662-439b-b7ee-bfeca28650da None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:21.370963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:21.383669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0b631-1662-439b-b7ee-bfeca28650da None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:21.389409 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27893414-26d0-4513-b607-f707a9596164 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:21.401434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:21.422256 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508f1bd-06fe-45c3-b75f-025c05fb1fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:21.423141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508f1bd-06fe-45c3-b75f-025c05fb1fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:21.426268 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508f1bd-06fe-45c3-b75f-025c05fb1fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:21.426268 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508f1bd-06fe-45c3-b75f-025c05fb1fc2 None admin] PATH_INFO: `/v3/groups/0d0b8234732741a4afb8034bd50a07fc` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:21.426268 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508f1bd-06fe-45c3-b75f-025c05fb1fc2 None admin] RBAC: Authorizing `identity:get_group(group_id=0d0b8234732741a4afb8034bd50a07fc)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:21.426780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508f1bd-06fe-45c3-b75f-025c05fb1fc2 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:21.428816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 198/412] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jun 15 10:02:21 2022] GET /identity/v3/groups/0d0b8234732741a4afb8034bd50a07fc => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:21.434251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508f1bd-06fe-45c3-b75f-025c05fb1fc2 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:21.465865 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee888b9-c87e-4b7a-84bc-552850b3531d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:21.467552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee888b9-c87e-4b7a-84bc-552850b3531d None admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:21.468254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee888b9-c87e-4b7a-84bc-552850b3531d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:21.469509 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee888b9-c87e-4b7a-84bc-552850b3531d None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e/groups/1b2ffec605c64ebda5b1c4f470c133a3/roles/d241ebe8f5654f46a5a92be6f6b0cc36` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:21.477814 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee888b9-c87e-4b7a-84bc-552850b3531d None admin] RBAC: Authorizing `identity:create_grant(project_id=354b4a7dec454bbe9294b154303e7f7e, group_id=1b2ffec605c64ebda5b1c4f470c133a3, role_id=d241ebe8f5654f46a5a92be6f6b0cc36)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:21.479410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee888b9-c87e-4b7a-84bc-552850b3531d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:21.553531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508f1bd-06fe-45c3-b75f-025c05fb1fc2 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:21.572347 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 215/413] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Jun 15 10:02:21 2022] PUT /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e/groups/1b2ffec605c64ebda5b1c4f470c133a3/roles/d241ebe8f5654f46a5a92be6f6b0cc36 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:02:21.593231 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fb620-0d97-4d03-89db-baf465429ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:21.594746 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fb620-0d97-4d03-89db-baf465429ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:21.594952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fb620-0d97-4d03-89db-baf465429ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:21.595235 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fb620-0d97-4d03-89db-baf465429ab9 None admin] PATH_INFO: `/v3/projects/8232a31392e948a78496aa9a0a9a81a0` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:21.596519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fb620-0d97-4d03-89db-baf465429ab9 None admin] RBAC: Authorizing `identity:get_project(project_id=8232a31392e948a78496aa9a0a9a81a0)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:21.598040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fb620-0d97-4d03-89db-baf465429ab9 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:21.600160 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 199/414] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:21 2022] GET /identity/v3/projects/8232a31392e948a78496aa9a0a9a81a0 => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:21.606081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee888b9-c87e-4b7a-84bc-552850b3531d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:21.735515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18413f-803a-456f-87ef-1154782a6513 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:21.735847 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18413f-803a-456f-87ef-1154782a6513 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:21.736161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18413f-803a-456f-87ef-1154782a6513 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:21.742233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 200/415] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:02:21.748337 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed63d09-fdd1-416a-b849-bd38c2498946 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:21.748660 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed63d09-fdd1-416a-b849-bd38c2498946 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:21.748947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed63d09-fdd1-416a-b849-bd38c2498946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:21.767876 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee888b9-c87e-4b7a-84bc-552850b3531d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:21.788201 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-4ed63d09-fdd1-416a-b849-bd38c2498946 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:21.789515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4ed63d09-fdd1-416a-b849-bd38c2498946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:21.807684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f889d-df6d-417a-851f-1d43fe24b3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:21.809097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f889d-df6d-417a-851f-1d43fe24b3b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:21.809619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f889d-df6d-417a-851f-1d43fe24b3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:21.810173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776f889d-df6d-417a-851f-1d43fe24b3b4 None admin] PATH_INFO: `/v3/projects/8232a31392e948a78496aa9a0a9a81a0/groups/0d0b8234732741a4afb8034bd50a07fc/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:21.813291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f889d-df6d-417a-851f-1d43fe24b3b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8232a31392e948a78496aa9a0a9a81a0, group_id=0d0b8234732741a4afb8034bd50a07fc, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:21.815132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776f889d-df6d-417a-851f-1d43fe24b3b4 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:21.821406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:21.871503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 201/416] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:02:21.896438 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed63d09-fdd1-416a-b849-bd38c2498946 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:21.918121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 216/417] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Jun 15 10:02:21 2022] PUT /identity/v3/projects/8232a31392e948a78496aa9a0a9a81a0/groups/0d0b8234732741a4afb8034bd50a07fc/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:02:22.010638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed63d09-fdd1-416a-b849-bd38c2498946 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:22.051767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d49a2-9d9c-4046-bb50-a8d561e16a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:22.052595 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d49a2-9d9c-4046-bb50-a8d561e16a11 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:22.052879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d49a2-9d9c-4046-bb50-a8d561e16a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:22.053186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d49a2-9d9c-4046-bb50-a8d561e16a11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:22.066515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d49a2-9d9c-4046-bb50-a8d561e16a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-b52d49a2-9d9c-4046-bb50-a8d561e16a11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:22.074940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:22.075884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:22.076662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:22.076662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:22.076662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:22.076662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:22.076662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:22.076662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:22.076662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:22.076662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:22.076662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:22.076662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:22.076662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:22.076662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:22.081083 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 202/418] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:02:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:22.084415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f889d-df6d-417a-851f-1d43fe24b3b4 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:22.128272 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:22.220244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776f889d-df6d-417a-851f-1d43fe24b3b4 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:22.259053 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e6333-6d32-482f-8b1a-902f920c5b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:22.259729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e6333-6d32-482f-8b1a-902f920c5b8a None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:22.260006 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e6333-6d32-482f-8b1a-902f920c5b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:22.260279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e6333-6d32-482f-8b1a-902f920c5b8a None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:22.261171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e6333-6d32-482f-8b1a-902f920c5b8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:22.261447 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e6333-6d32-482f-8b1a-902f920c5b8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:22.262808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e6333-6d32-482f-8b1a-902f920c5b8a None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:22.273557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 217/419] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:22.279173 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d49a2-9d9c-4046-bb50-a8d561e16a11 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:22.299818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:22.414963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d49a2-9d9c-4046-bb50-a8d561e16a11 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:22.463979 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a1433-c235-4bc2-9807-bc16413e6a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:22.464661 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a1433-c235-4bc2-9807-bc16413e6a0d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:22.464927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a1433-c235-4bc2-9807-bc16413e6a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:22.465231 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a1433-c235-4bc2-9807-bc16413e6a0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:22.566329 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a1433-c235-4bc2-9807-bc16413e6a0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:22.567820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a1433-c235-4bc2-9807-bc16413e6a0d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:22.591551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:22.698996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 203/420] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:22.705233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e6333-6d32-482f-8b1a-902f920c5b8a None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:22.824358 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:22.832711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085c6bb-2098-49bd-a6ce-a152939d5f65 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:22.839282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085c6bb-2098-49bd-a6ce-a152939d5f65 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:22.839282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085c6bb-2098-49bd-a6ce-a152939d5f65 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:22.839282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 204/421] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:22.839965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c618fe2-6022-4eaf-884f-befd496635fe None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:22.840161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e6333-6d32-482f-8b1a-902f920c5b8a None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:22.840271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c618fe2-6022-4eaf-884f-befd496635fe None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:22.840521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c618fe2-6022-4eaf-884f-befd496635fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:22.878227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-5c618fe2-6022-4eaf-884f-befd496635fe None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:22.879432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-5c618fe2-6022-4eaf-884f-befd496635fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:22.886963 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6f3e-2b07-4abd-9e8a-813945105e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:22.887846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6f3e-2b07-4abd-9e8a-813945105e9c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:22.888129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6f3e-2b07-4abd-9e8a-813945105e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:22.888403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6f3e-2b07-4abd-9e8a-813945105e9c None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:22.894567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef6f3e-2b07-4abd-9e8a-813945105e9c None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:22.896037 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef6f3e-2b07-4abd-9e8a-813945105e9c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:22.898806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 218/422] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:22 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:22.904804 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6f3e-2b07-4abd-9e8a-813945105e9c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:23.026740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 205/423] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:23.054924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6f3e-2b07-4abd-9e8a-813945105e9c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:23.061163 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c618fe2-6022-4eaf-884f-befd496635fe None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:23.101623 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304a059-8238-4325-9789-2155437786f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:23.103157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304a059-8238-4325-9789-2155437786f7 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:23.103643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304a059-8238-4325-9789-2155437786f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:23.104116 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304a059-8238-4325-9789-2155437786f7 None admin] PATH_INFO: `/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:23.105652 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304a059-8238-4325-9789-2155437786f7 None admin] RBAC: Authorizing `identity:get_group(group_id=1b2ffec605c64ebda5b1c4f470c133a3)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:23.109154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2304a059-8238-4325-9789-2155437786f7 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:23.114817 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 219/424] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jun 15 10:02:22 2022] GET /identity/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:23.135981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304a059-8238-4325-9789-2155437786f7 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:23.161377 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:23.172334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:23.270440 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304a059-8238-4325-9789-2155437786f7 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:23.271980 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c618fe2-6022-4eaf-884f-befd496635fe None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:23.309107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb36894-9241-4d69-8903-21b7728f9319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:23.309939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb36894-9241-4d69-8903-21b7728f9319 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:23.310429 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb36894-9241-4d69-8903-21b7728f9319 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:23.310604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb36894-9241-4d69-8903-21b7728f9319 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:23.316689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2a58c-43aa-47ab-9468-bde7f0f9db38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:23.317895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2a58c-43aa-47ab-9468-bde7f0f9db38 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:23.318538 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2a58c-43aa-47ab-9468-bde7f0f9db38 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:23.318992 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2a58c-43aa-47ab-9468-bde7f0f9db38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:23.320731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2a58c-43aa-47ab-9468-bde7f0f9db38 None admin] RBAC: Adding query filter params (group.id=1b2ffec605c64ebda5b1c4f470c133a3, role.id=d241ebe8f5654f46a5a92be6f6b0cc36, scope.project.id=2710654d490f43b0adadfd7a62df84bd) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:23.321276 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2a58c-43aa-47ab-9468-bde7f0f9db38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:23.323187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2a58c-43aa-47ab-9468-bde7f0f9db38 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:23.323520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb36894-9241-4d69-8903-21b7728f9319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:23.331103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 220/425] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jun 15 10:02:23 2022] GET /identity/v3/role_assignments?group.id=1b2ffec605c64ebda5b1c4f470c133a3&scope.project.id=2710654d490f43b0adadfd7a62df84bd&role.id=d241ebe8f5654f46a5a92be6f6b0cc36 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-3eb36894-9241-4d69-8903-21b7728f9319 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:23.332567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:23.333459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:23.334249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:23.334249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:23.334249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:23.334249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:23.334249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:23.334249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:23.334249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:23.334249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:23.334249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:23.334249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:23.334249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:23.334249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:23.334249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 206/426] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:23.335679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c95bd-8bcc-4d9d-9f22-f24741e9a72f None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:23.336036 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c95bd-8bcc-4d9d-9f22-f24741e9a72f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:23.336318 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c95bd-8bcc-4d9d-9f22-f24741e9a72f None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:23.345586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 221/427] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:23.350463 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dc8af-7901-4bd1-aadf-29082564d576 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:23.351263 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb36894-9241-4d69-8903-21b7728f9319 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:23.352220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dc8af-7901-4bd1-aadf-29082564d576 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:23.353098 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785dc8af-7901-4bd1-aadf-29082564d576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:23.395881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-785dc8af-7901-4bd1-aadf-29082564d576 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:23.398258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-785dc8af-7901-4bd1-aadf-29082564d576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:23.547001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb36894-9241-4d69-8903-21b7728f9319 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:23.549149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:23.570471 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 222/428] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:23.599507 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f696b-2cb1-46c5-ba07-9fd72a8a671f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:23.600194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f696b-2cb1-46c5-ba07-9fd72a8a671f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:23.600533 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f696b-2cb1-46c5-ba07-9fd72a8a671f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:23.600818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f696b-2cb1-46c5-ba07-9fd72a8a671f None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:23.601915 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f696b-2cb1-46c5-ba07-9fd72a8a671f None admin] RBAC: Adding query filter params (name=member) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:23.602288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f696b-2cb1-46c5-ba07-9fd72a8a671f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:23.603635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f696b-2cb1-46c5-ba07-9fd72a8a671f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:23.606086 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dc8af-7901-4bd1-aadf-29082564d576 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:23.616281 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 207/429] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:02:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:23.621843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f696b-2cb1-46c5-ba07-9fd72a8a671f None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:23.640688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:23.748067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785dc8af-7901-4bd1-aadf-29082564d576 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:23.768267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f696b-2cb1-46c5-ba07-9fd72a8a671f None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:23.788501 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c11b1-dc28-4b1f-9637-15d4a17c8324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:23.789873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c11b1-dc28-4b1f-9637-15d4a17c8324 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:23.790716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c11b1-dc28-4b1f-9637-15d4a17c8324 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:23.791166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c11b1-dc28-4b1f-9637-15d4a17c8324 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:23.807507 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c11b1-dc28-4b1f-9637-15d4a17c8324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:23.808530 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b93cd4-1f2d-42ad-885e-680fd0ecf269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:23.809228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b93cd4-1f2d-42ad-885e-680fd0ecf269 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:23.809509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b93cd4-1f2d-42ad-885e-680fd0ecf269 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:23.809785 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b93cd4-1f2d-42ad-885e-680fd0ecf269 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-311c11b1-dc28-4b1f-9637-15d4a17c8324 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:23.818777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:23.819898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:23.820843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:23.820843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:23.820843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:23.820843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:23.820843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:23.820843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:23.820843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:23.820843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:23.820843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:23.820843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:23.820843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 15 10:02:23.820843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:23.823533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 223/430] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:23.830202 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c11b1-dc28-4b1f-9637-15d4a17c8324 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:23.859232 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:23.907564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b93cd4-1f2d-42ad-885e-680fd0ecf269 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:23.909446 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b93cd4-1f2d-42ad-885e-680fd0ecf269 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:23.938288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:23.954007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c11b1-dc28-4b1f-9637-15d4a17c8324 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:23.996676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647e1a8-7168-4a37-a084-10eb971e50fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:23.997637 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647e1a8-7168-4a37-a084-10eb971e50fa None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:23.997921 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647e1a8-7168-4a37-a084-10eb971e50fa None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:23.998193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647e1a8-7168-4a37-a084-10eb971e50fa None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:23.998947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647e1a8-7168-4a37-a084-10eb971e50fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:23.999249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647e1a8-7168-4a37-a084-10eb971e50fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:24.002340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647e1a8-7168-4a37-a084-10eb971e50fa None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:24.016620 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 224/431] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:02:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:24.018122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647e1a8-7168-4a37-a084-10eb971e50fa None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:24.044600 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 208/432] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:24.045690 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b93cd4-1f2d-42ad-885e-680fd0ecf269 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:24.123580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:24.166033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647e1a8-7168-4a37-a084-10eb971e50fa None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:24.176934 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b93cd4-1f2d-42ad-885e-680fd0ecf269 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:24.209185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392c19d-0a2b-4768-882a-7c22cf34ef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:24.209850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c19d-0a2b-4768-882a-7c22cf34ef54 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:24.210137 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c19d-0a2b-4768-882a-7c22cf34ef54 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:24.210415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c19d-0a2b-4768-882a-7c22cf34ef54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:24.216601 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad96bc-04b9-41de-b831-32836bc81c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:24.217506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad96bc-04b9-41de-b831-32836bc81c02 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:24.217790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad96bc-04b9-41de-b831-32836bc81c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:24.218649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad96bc-04b9-41de-b831-32836bc81c02 None admin] PATH_INFO: `/v3/projects/2710654d490f43b0adadfd7a62df84bd` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:24.219919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad96bc-04b9-41de-b831-32836bc81c02 None admin] RBAC: Authorizing `identity:get_project(project_id=2710654d490f43b0adadfd7a62df84bd)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:24.221420 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad96bc-04b9-41de-b831-32836bc81c02 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:24.223481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 209/433] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:24 2022] GET /identity/v3/projects/2710654d490f43b0adadfd7a62df84bd => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:24.223888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:24.224832 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633b074-505e-4a35-bbc1-1909aef94e19 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:24.225083 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633b074-505e-4a35-bbc1-1909aef94e19 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:24.225451 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633b074-505e-4a35-bbc1-1909aef94e19 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:24.226804 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 210/434] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:24.229859 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633b074-505e-4a35-bbc1-1909aef94e19 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:24.291802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392c19d-0a2b-4768-882a-7c22cf34ef54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:24.293419 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392c19d-0a2b-4768-882a-7c22cf34ef54 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:24.344988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633b074-505e-4a35-bbc1-1909aef94e19 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:24.381870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12aaaff-b827-45ad-a39c-8a287a7aec0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:24.382655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12aaaff-b827-45ad-a39c-8a287a7aec0a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:24.382943 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12aaaff-b827-45ad-a39c-8a287a7aec0a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:24.383221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12aaaff-b827-45ad-a39c-8a287a7aec0a None admin] PATH_INFO: `/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:24.384432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12aaaff-b827-45ad-a39c-8a287a7aec0a None admin] RBAC: Authorizing `identity:get_group(group_id=1b2ffec605c64ebda5b1c4f470c133a3)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:24.385935 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12aaaff-b827-45ad-a39c-8a287a7aec0a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:24.387745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 211/435] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jun 15 10:02:24 2022] GET /identity/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:24.388813 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21cca8-0f6b-44eb-9a64-18b9accc032e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:24.389125 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21cca8-0f6b-44eb-9a64-18b9accc032e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:24.389404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21cca8-0f6b-44eb-9a64-18b9accc032e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:24.396794 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:24.410641 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-7d21cca8-0f6b-44eb-9a64-18b9accc032e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:24.415828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-7d21cca8-0f6b-44eb-9a64-18b9accc032e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:24.415949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 225/436] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 10:02:24.415949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392c19d-0a2b-4768-882a-7c22cf34ef54 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:24.461783 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:24.495422 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 212/437] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:24.502846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21cca8-0f6b-44eb-9a64-18b9accc032e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:24.532633 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392c19d-0a2b-4768-882a-7c22cf34ef54 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:24.572293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8d8d9-d915-4312-bdfb-b5dde9066e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:24.573009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8d8d9-d915-4312-bdfb-b5dde9066e4a None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:24.573407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8d8d9-d915-4312-bdfb-b5dde9066e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:24.573598 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8d8d9-d915-4312-bdfb-b5dde9066e4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:24.574442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8d8d9-d915-4312-bdfb-b5dde9066e4a None admin] RBAC: Adding query filter params (group.id=1b2ffec605c64ebda5b1c4f470c133a3, role.id=c91413ec3feb454ca7186eb52a57ed96, scope.project.id=2710654d490f43b0adadfd7a62df84bd) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:24.574753 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8d8d9-d915-4312-bdfb-b5dde9066e4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:24.576233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8d8d9-d915-4312-bdfb-b5dde9066e4a None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:24.584143 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 226/438] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jun 15 10:02:24 2022] GET /identity/v3/role_assignments?group.id=1b2ffec605c64ebda5b1c4f470c133a3&scope.project.id=2710654d490f43b0adadfd7a62df84bd&role.id=c91413ec3feb454ca7186eb52a57ed96 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:24.584812 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8d8d9-d915-4312-bdfb-b5dde9066e4a None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:24.647839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21cca8-0f6b-44eb-9a64-18b9accc032e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:24.689376 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:24.695283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ae885-b9d5-4476-8aff-b9226d3f2eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:24.696351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ae885-b9d5-4476-8aff-b9226d3f2eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:24.704341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ae885-b9d5-4476-8aff-b9226d3f2eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:24.704341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ae885-b9d5-4476-8aff-b9226d3f2eb5 None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:24.704341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ae885-b9d5-4476-8aff-b9226d3f2eb5 None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:24.704341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ae885-b9d5-4476-8aff-b9226d3f2eb5 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:24.708252 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 213/439] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:24 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:24.709796 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ae885-b9d5-4476-8aff-b9226d3f2eb5 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:24.727304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8d8d9-d915-4312-bdfb-b5dde9066e4a None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:24.731703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:24.777788 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d7963-0937-425d-a42b-ed76651545b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:24.778614 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d7963-0937-425d-a42b-ed76651545b3 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:24.778896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d7963-0937-425d-a42b-ed76651545b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:24.779173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d7963-0937-425d-a42b-ed76651545b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:24.790984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d7963-0937-425d-a42b-ed76651545b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-131d7963-0937-425d-a42b-ed76651545b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:24.799939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:24.801270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:24.802067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:24.802067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:24.802067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:24.802067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:24.802067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:24.802067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:24.802067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:24.802067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:24.802067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:24.802067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:24.802067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:24.802067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:24.802067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 227/440] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:02:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:24.803127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36986585-3609-4743-96f1-9d907d2211c0 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:24.803416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36986585-3609-4743-96f1-9d907d2211c0 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:24.803700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36986585-3609-4743-96f1-9d907d2211c0 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:24.805157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 228/441] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:24.808068 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36986585-3609-4743-96f1-9d907d2211c0 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:24.839161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ae885-b9d5-4476-8aff-b9226d3f2eb5 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:24.880601 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d08662-cbbc-4e57-a51b-d71c49921719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:24.881332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d08662-cbbc-4e57-a51b-d71c49921719 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:24.881618 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d08662-cbbc-4e57-a51b-d71c49921719 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:24.881890 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d08662-cbbc-4e57-a51b-d71c49921719 None admin] PATH_INFO: `/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:24.883140 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d08662-cbbc-4e57-a51b-d71c49921719 None admin] RBAC: Authorizing `identity:get_group(group_id=1b2ffec605c64ebda5b1c4f470c133a3)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:24.884597 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d08662-cbbc-4e57-a51b-d71c49921719 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:24.886672 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 214/442] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jun 15 10:02:24 2022] GET /identity/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:24.887760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d46456-aefd-4d14-b589-424485e46bd5 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:24.888051 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d46456-aefd-4d14-b589-424485e46bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:24.888329 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d46456-aefd-4d14-b589-424485e46bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:24.908022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-76d46456-aefd-4d14-b589-424485e46bd5 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:24.909180 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-76d46456-aefd-4d14-b589-424485e46bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:24.931220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36986585-3609-4743-96f1-9d907d2211c0 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:24.972079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ff289-fc06-4a87-a184-7913024a3231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:24.972079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ff289-fc06-4a87-a184-7913024a3231 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:24.972079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ff289-fc06-4a87-a184-7913024a3231 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:24.972426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ff289-fc06-4a87-a184-7913024a3231 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:24.973175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ff289-fc06-4a87-a184-7913024a3231 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:24.973489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ff289-fc06-4a87-a184-7913024a3231 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:24.974899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ff289-fc06-4a87-a184-7913024a3231 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:24.984887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:24.985350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 229/443] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:24.986598 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ff289-fc06-4a87-a184-7913024a3231 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:24.996982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 215/444] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:24.996982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d46456-aefd-4d14-b589-424485e46bd5 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:25.022618 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:25.116784 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ff289-fc06-4a87-a184-7913024a3231 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:25.119008 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d46456-aefd-4d14-b589-424485e46bd5 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:25.162484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f0281-969e-48fe-a83f-258914e08791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:25.163067 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f0281-969e-48fe-a83f-258914e08791 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:25.163358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f0281-969e-48fe-a83f-258914e08791 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:25.166730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f0281-969e-48fe-a83f-258914e08791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:25.166845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d19c6-4046-4a3a-b58a-8433596c974d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:25.166845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d19c6-4046-4a3a-b58a-8433596c974d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:25.166845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d19c6-4046-4a3a-b58a-8433596c974d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:25.166845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d19c6-4046-4a3a-b58a-8433596c974d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:25.167488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d19c6-4046-4a3a-b58a-8433596c974d None admin] RBAC: Adding query filter params (group.id=1b2ffec605c64ebda5b1c4f470c133a3, role.id=c91413ec3feb454ca7186eb52a57ed96, scope.project.id=354b4a7dec454bbe9294b154303e7f7e) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:25.167488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d19c6-4046-4a3a-b58a-8433596c974d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:25.170829 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d19c6-4046-4a3a-b58a-8433596c974d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:25.176562 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 230/445] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jun 15 10:02:24 2022] GET /identity/v3/role_assignments?group.id=1b2ffec605c64ebda5b1c4f470c133a3&scope.project.id=354b4a7dec454bbe9294b154303e7f7e&role.id=c91413ec3feb454ca7186eb52a57ed96 => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:25.177562 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d19c6-4046-4a3a-b58a-8433596c974d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:25.253739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:25.257656 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f0281-969e-48fe-a83f-258914e08791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:25.260160 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f0281-969e-48fe-a83f-258914e08791 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:25.298081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:25.308571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d19c6-4046-4a3a-b58a-8433596c974d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:25.352362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f5d35-7f41-4b8a-965c-e7e79fbf1eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:25.352607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f5d35-7f41-4b8a-965c-e7e79fbf1eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:25.352892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f5d35-7f41-4b8a-965c-e7e79fbf1eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:25.353237 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f5d35-7f41-4b8a-965c-e7e79fbf1eb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:25.365898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f5d35-7f41-4b8a-965c-e7e79fbf1eb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-ef3f5d35-7f41-4b8a-965c-e7e79fbf1eb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:25.373825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:25.374918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:25.375694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:25.375694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:25.375694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:25.375694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:25.375694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:25.375694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:25.375694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:25.375694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:25.375694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:25.375694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:25.375694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:02:25.375694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:25.375694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 231/446] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:02:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:25.380872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f5d35-7f41-4b8a-965c-e7e79fbf1eb7 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:25.383441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 216/447] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:25.389115 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f0281-969e-48fe-a83f-258914e08791 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:25.501853 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f5d35-7f41-4b8a-965c-e7e79fbf1eb7 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:25.508029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:25.513201 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f0281-969e-48fe-a83f-258914e08791 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:25.541744 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4175f9-f49f-4c93-a1be-e70c40351c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:25.542283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4175f9-f49f-4c93-a1be-e70c40351c72 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:25.542634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4175f9-f49f-4c93-a1be-e70c40351c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:25.542911 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4175f9-f49f-4c93-a1be-e70c40351c72 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:25.543672 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4175f9-f49f-4c93-a1be-e70c40351c72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:25.543978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4175f9-f49f-4c93-a1be-e70c40351c72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:25.545457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4175f9-f49f-4c93-a1be-e70c40351c72 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:25.556073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 232/448] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:25.562684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4175f9-f49f-4c93-a1be-e70c40351c72 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:25.570284 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb367e6b-ede2-4f8e-9ba8-07545e738069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:25.571182 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb367e6b-ede2-4f8e-9ba8-07545e738069 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:25.571437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb367e6b-ede2-4f8e-9ba8-07545e738069 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:25.571709 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb367e6b-ede2-4f8e-9ba8-07545e738069 None admin] PATH_INFO: `/v3/projects/354b4a7dec454bbe9294b154303e7f7e` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:25.572944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb367e6b-ede2-4f8e-9ba8-07545e738069 None admin] RBAC: Authorizing `identity:get_project(project_id=354b4a7dec454bbe9294b154303e7f7e)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:25.574450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb367e6b-ede2-4f8e-9ba8-07545e738069 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:25.576207 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 217/449] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:25 2022] GET /identity/v3/projects/354b4a7dec454bbe9294b154303e7f7e => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:25.580437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:25.582942 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb367e6b-ede2-4f8e-9ba8-07545e738069 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:25.670820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4175f9-f49f-4c93-a1be-e70c40351c72 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:25.688421 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb367e6b-ede2-4f8e-9ba8-07545e738069 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:25.706578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1f13f-7eb3-4fe7-8db0-f099e6b62449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:25.707249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1f13f-7eb3-4fe7-8db0-f099e6b62449 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:25.707534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1f13f-7eb3-4fe7-8db0-f099e6b62449 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:25.707851 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1f13f-7eb3-4fe7-8db0-f099e6b62449 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:25.723974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f845d8-2bbf-406f-ac25-c352c3285c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:25.724845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f845d8-2bbf-406f-ac25-c352c3285c23 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:25.725150 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f845d8-2bbf-406f-ac25-c352c3285c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:25.725421 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f845d8-2bbf-406f-ac25-c352c3285c23 None admin] PATH_INFO: `/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:25.726633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f845d8-2bbf-406f-ac25-c352c3285c23 None admin] RBAC: Authorizing `identity:get_group(group_id=1b2ffec605c64ebda5b1c4f470c133a3)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:25.728075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f845d8-2bbf-406f-ac25-c352c3285c23 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:25.729736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 218/450] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jun 15 10:02:25 2022] GET /identity/v3/groups/1b2ffec605c64ebda5b1c4f470c133a3 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:25.735889 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f845d8-2bbf-406f-ac25-c352c3285c23 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:25.770633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:25.778439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1f13f-7eb3-4fe7-8db0-f099e6b62449 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:25.779923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1f13f-7eb3-4fe7-8db0-f099e6b62449 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:25.793767 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:25.838399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f845d8-2bbf-406f-ac25-c352c3285c23 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:25.873241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7ad10-63a5-46ef-b49b-3d8039773ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:25.873893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7ad10-63a5-46ef-b49b-3d8039773ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:25.874178 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7ad10-63a5-46ef-b49b-3d8039773ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:25.874559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7ad10-63a5-46ef-b49b-3d8039773ef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:25.875590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7ad10-63a5-46ef-b49b-3d8039773ef4 None admin] RBAC: Adding query filter params (group.id=1b2ffec605c64ebda5b1c4f470c133a3, role.id=d241ebe8f5654f46a5a92be6f6b0cc36, scope.project.id=354b4a7dec454bbe9294b154303e7f7e) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:25.875909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7ad10-63a5-46ef-b49b-3d8039773ef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:25.877398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7ad10-63a5-46ef-b49b-3d8039773ef4 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:25.881258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 233/451] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 15 10:02:25.886867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1f13f-7eb3-4fe7-8db0-f099e6b62449 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:25.900989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 219/452] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jun 15 10:02:25 2022] GET /identity/v3/role_assignments?group.id=1b2ffec605c64ebda5b1c4f470c133a3&scope.project.id=354b4a7dec454bbe9294b154303e7f7e&role.id=d241ebe8f5654f46a5a92be6f6b0cc36 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:26.006935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1f13f-7eb3-4fe7-8db0-f099e6b62449 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:26.024859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:26.051668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db0bdd-5d50-4f88-9209-c514caa93e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:26.052687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db0bdd-5d50-4f88-9209-c514caa93e49 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:26.052991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db0bdd-5d50-4f88-9209-c514caa93e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:26.053273 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db0bdd-5d50-4f88-9209-c514caa93e49 None admin] PATH_INFO: `/v3/projects/8232a31392e948a78496aa9a0a9a81a0` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:26.054556 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db0bdd-5d50-4f88-9209-c514caa93e49 None admin] RBAC: Authorizing `identity:get_project(project_id=8232a31392e948a78496aa9a0a9a81a0)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:26.056051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db0bdd-5d50-4f88-9209-c514caa93e49 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:26.058153 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 234/453] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:02:25 2022] GET /identity/v3/projects/8232a31392e948a78496aa9a0a9a81a0 => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:26.064752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7ad10-63a5-46ef-b49b-3d8039773ef4 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:26.184509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:26.191528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7ad10-63a5-46ef-b49b-3d8039773ef4 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:26.235478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18a3a3-db16-4f11-94fc-2a3bde94347e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:26.236355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a3a3-db16-4f11-94fc-2a3bde94347e None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:26.236634 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a3a3-db16-4f11-94fc-2a3bde94347e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:26.236902 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a3a3-db16-4f11-94fc-2a3bde94347e None admin] PATH_INFO: `/v3/groups/0d0b8234732741a4afb8034bd50a07fc` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:26.238757 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18a3a3-db16-4f11-94fc-2a3bde94347e None admin] RBAC: Authorizing `identity:get_group(group_id=0d0b8234732741a4afb8034bd50a07fc)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:26.240192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18a3a3-db16-4f11-94fc-2a3bde94347e None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:26.242205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 220/454] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jun 15 10:02:26 2022] GET /identity/v3/groups/0d0b8234732741a4afb8034bd50a07fc => generated 229 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:26.248036 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db0bdd-5d50-4f88-9209-c514caa93e49 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:26.354753 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db0bdd-5d50-4f88-9209-c514caa93e49 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:26.389100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeefe0c-0665-4e35-a3c2-f7d52a8a1656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:26.389751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeefe0c-0665-4e35-a3c2-f7d52a8a1656 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:26.390079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeefe0c-0665-4e35-a3c2-f7d52a8a1656 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:26.390335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeefe0c-0665-4e35-a3c2-f7d52a8a1656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:26.391165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeefe0c-0665-4e35-a3c2-f7d52a8a1656 None admin] RBAC: Adding query filter params (group.id=0d0b8234732741a4afb8034bd50a07fc, role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=8232a31392e948a78496aa9a0a9a81a0) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:26.391476 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeefe0c-0665-4e35-a3c2-f7d52a8a1656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:26.392906 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeefe0c-0665-4e35-a3c2-f7d52a8a1656 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:26.398643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 235/455] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jun 15 10:02:26 2022] GET /identity/v3/role_assignments?group.id=0d0b8234732741a4afb8034bd50a07fc&scope.project.id=8232a31392e948a78496aa9a0a9a81a0&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:28.916113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597dd30-88eb-4e9c-87e3-9ef7ed13b72a None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:28.916598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597dd30-88eb-4e9c-87e3-9ef7ed13b72a None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:28.916598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597dd30-88eb-4e9c-87e3-9ef7ed13b72a None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:28.918401 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 221/456] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:28.924548 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858affa-35bb-4331-aac8-6284f108d479 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:28.925452 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858affa-35bb-4331-aac8-6284f108d479 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:28.926185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858affa-35bb-4331-aac8-6284f108d479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:28.954120 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-f858affa-35bb-4331-aac8-6284f108d479 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:28.955691 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f858affa-35bb-4331-aac8-6284f108d479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:29.010534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:29.051729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 236/457] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:29.098510 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597dd30-88eb-4e9c-87e3-9ef7ed13b72a None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:29.236200 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735552f-68f7-4a34-88a2-36f9276f56bb None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:29.236917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735552f-68f7-4a34-88a2-36f9276f56bb None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:29.237454 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735552f-68f7-4a34-88a2-36f9276f56bb None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:29.242343 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597dd30-88eb-4e9c-87e3-9ef7ed13b72a None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:29.242640 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 237/458] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:29.245282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c662f7-6abb-4f16-8fdf-6e37652630f9 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:29.245839 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c662f7-6abb-4f16-8fdf-6e37652630f9 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:29.246312 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c662f7-6abb-4f16-8fdf-6e37652630f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:29.283894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-91c662f7-6abb-4f16-8fdf-6e37652630f9 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:29.283894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-91c662f7-6abb-4f16-8fdf-6e37652630f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:29.290597 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923c07b-18de-45f4-921a-bb5229e5f46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:29.291416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923c07b-18de-45f4-921a-bb5229e5f46d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:29.291697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923c07b-18de-45f4-921a-bb5229e5f46d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:29.291979 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a923c07b-18de-45f4-921a-bb5229e5f46d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:29.322858 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923c07b-18de-45f4-921a-bb5229e5f46d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:29.325850 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a923c07b-18de-45f4-921a-bb5229e5f46d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:29.365308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-a923c07b-18de-45f4-921a-bb5229e5f46d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:29.365308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:29.365308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:29.365308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:29.365308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:29.365308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:29.365308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:29.365308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:29.365308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:29.365308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:29.365308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:29.365308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:29.365308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:29.365308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:29.365308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:29.366208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:29.367044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:29.367044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:29.367044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:29.367044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:29.367044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 222/459] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:29.373917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923c07b-18de-45f4-921a-bb5229e5f46d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:29.406055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 238/460] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:29.411278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:29.431965 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c662f7-6abb-4f16-8fdf-6e37652630f9 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:29.568222 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c662f7-6abb-4f16-8fdf-6e37652630f9 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:29.571189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:29.622082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1193c-3ef0-4dd6-b50c-94ea7be7144f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:29.624432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a923c07b-18de-45f4-921a-bb5229e5f46d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:29.626212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1193c-3ef0-4dd6-b50c-94ea7be7144f None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:29.626720 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1193c-3ef0-4dd6-b50c-94ea7be7144f None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:29.629432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1193c-3ef0-4dd6-b50c-94ea7be7144f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:29.655251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1193c-3ef0-4dd6-b50c-94ea7be7144f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:29.660680 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1193c-3ef0-4dd6-b50c-94ea7be7144f None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:29.689558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-5dd1193c-3ef0-4dd6-b50c-94ea7be7144f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:29.689558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:29.689558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:29.689558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:29.689558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:29.689558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:29.689558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:29.689558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:29.689558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:29.689558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:29.689558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:29.689558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:29.689558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:29.689558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:29.689558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:29.690530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:29.691366 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:29.691366 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:29.691366 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:29.691366 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:29.693838 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 239/461] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:29.695427 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015442f7-926a-40e0-98ac-ca5a70f9da13 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:29.695856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015442f7-926a-40e0-98ac-ca5a70f9da13 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:29.696296 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015442f7-926a-40e0-98ac-ca5a70f9da13 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:29.698010 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 240/462] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:29.699847 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015442f7-926a-40e0-98ac-ca5a70f9da13 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:29.710462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382f12c-6900-4a01-8fb4-4689137dacf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:29.711130 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382f12c-6900-4a01-8fb4-4689137dacf0 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:29.711599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382f12c-6900-4a01-8fb4-4689137dacf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:29.711882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382f12c-6900-4a01-8fb4-4689137dacf0 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:29.712622 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382f12c-6900-4a01-8fb4-4689137dacf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:29.712930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382f12c-6900-4a01-8fb4-4689137dacf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:29.714331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382f12c-6900-4a01-8fb4-4689137dacf0 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:29.726185 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 223/463] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:29.727448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f796c-3cb5-4867-8d85-24a451a403a9 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:29.727735 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f796c-3cb5-4867-8d85-24a451a403a9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:29.728010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f796c-3cb5-4867-8d85-24a451a403a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:29.746353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-105f796c-3cb5-4867-8d85-24a451a403a9 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:29.747588 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-105f796c-3cb5-4867-8d85-24a451a403a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:29.818111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015442f7-926a-40e0-98ac-ca5a70f9da13 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:29.824849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 224/464] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:29.824849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c73531-fc6f-4803-be09-abf8c5b22d81 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:29.824849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c73531-fc6f-4803-be09-abf8c5b22d81 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:29.824849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c73531-fc6f-4803-be09-abf8c5b22d81 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:29.824849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 225/465] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:29.824849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c73531-fc6f-4803-be09-abf8c5b22d81 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:29.840648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:29.858082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b7215-e106-4690-9e17-49ff35cbd4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:29.859050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b7215-e106-4690-9e17-49ff35cbd4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:29.859417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b7215-e106-4690-9e17-49ff35cbd4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:29.859852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b7215-e106-4690-9e17-49ff35cbd4c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:29.860590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b7215-e106-4690-9e17-49ff35cbd4c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:29.860940 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b7215-e106-4690-9e17-49ff35cbd4c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:29.867655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b7215-e106-4690-9e17-49ff35cbd4c2 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:29.879361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 241/466] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:29.879361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61894b-61a1-437a-b693-6a270b9dc9a8 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:29.879361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61894b-61a1-437a-b693-6a270b9dc9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:29.879361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61894b-61a1-437a-b693-6a270b9dc9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:29.895607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-6c61894b-61a1-437a-b693-6a270b9dc9a8 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:29.896815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6c61894b-61a1-437a-b693-6a270b9dc9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:29.908120 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:29.950708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c73531-fc6f-4803-be09-abf8c5b22d81 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:29.968315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 242/467] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:29.969773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61894b-61a1-437a-b693-6a270b9dc9a8 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:30.004254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e45df-40f6-456a-a3b5-40466a5f1ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:30.004923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e45df-40f6-456a-a3b5-40466a5f1ed4 None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:30.005232 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e45df-40f6-456a-a3b5-40466a5f1ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:30.005499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e45df-40f6-456a-a3b5-40466a5f1ed4 None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:30.006208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e45df-40f6-456a-a3b5-40466a5f1ed4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:30.007654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e45df-40f6-456a-a3b5-40466a5f1ed4 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:30.066701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 226/468] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:02:29 2022] POST /identity/v3/users => generated 246 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:02:30.067513 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e45df-40f6-456a-a3b5-40466a5f1ed4 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:30.091324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61894b-61a1-437a-b693-6a270b9dc9a8 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:30.151240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85892abe-2d2d-42db-814f-4e4af32e3332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:30.152450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:30.153926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85892abe-2d2d-42db-814f-4e4af32e3332 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:30.154536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85892abe-2d2d-42db-814f-4e4af32e3332 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:30.154995 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85892abe-2d2d-42db-814f-4e4af32e3332 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:30.176616 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85892abe-2d2d-42db-814f-4e4af32e3332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:30.178710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85892abe-2d2d-42db-814f-4e4af32e3332 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:30.190961 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:30.204485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e45df-40f6-456a-a3b5-40466a5f1ed4 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:30.209031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-85892abe-2d2d-42db-814f-4e4af32e3332 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:30.209031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:30.209031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:30.209031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:30.209031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:30.209031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:30.209031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:30.209031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:30.209031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:30.209031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:30.209031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:30.209031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:30.209031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:30.209031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:30.209031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:30.209913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:30.210762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:30.210762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:30.210762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:30.210762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:30.212652 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 243/469] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:30.214407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85892abe-2d2d-42db-814f-4e4af32e3332 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:30.252905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b5f88-0184-4c18-b152-d97b6a13aecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:30.254029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b5f88-0184-4c18-b152-d97b6a13aecb None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:30.254378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b5f88-0184-4c18-b152-d97b6a13aecb None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:30.254687 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b5f88-0184-4c18-b152-d97b6a13aecb None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:30.256881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b5f88-0184-4c18-b152-d97b6a13aecb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:30.256881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b5f88-0184-4c18-b152-d97b6a13aecb None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:30.332555 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85892abe-2d2d-42db-814f-4e4af32e3332 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:30.341143 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 227/470] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:02:30 2022] POST /identity/v3/users => generated 248 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:02:30.342595 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b5f88-0184-4c18-b152-d97b6a13aecb None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:30.445032 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a3c5b-1cf6-4b93-b833-c12f9e132115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:30.445930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a3c5b-1cf6-4b93-b833-c12f9e132115 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:30.446247 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a3c5b-1cf6-4b93-b833-c12f9e132115 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:30.446530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a3c5b-1cf6-4b93-b833-c12f9e132115 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:30.459122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a3c5b-1cf6-4b93-b833-c12f9e132115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:30.460651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a3c5b-1cf6-4b93-b833-c12f9e132115 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:30.480518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-d79a3c5b-1cf6-4b93-b833-c12f9e132115 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:30.480518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:30.480518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:30.480518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:30.480518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:30.480518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:30.480518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:30.480518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:30.480518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:30.480518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:30.480518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:30.480518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:30.480518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:30.480518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:30.480518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:30.481493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:30.482310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:30.482310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:30.482310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:30.482310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:30.484847 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b5f88-0184-4c18-b152-d97b6a13aecb None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:30.487331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 244/471] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:30.488668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a3c5b-1cf6-4b93-b833-c12f9e132115 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:30.510774 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:30.532227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb078418-5c46-42b3-b5c2-eef6c970e132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:30.537276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb078418-5c46-42b3-b5c2-eef6c970e132 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:30.537276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb078418-5c46-42b3-b5c2-eef6c970e132 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:30.537276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb078418-5c46-42b3-b5c2-eef6c970e132 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:30.537276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb078418-5c46-42b3-b5c2-eef6c970e132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:30.537276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb078418-5c46-42b3-b5c2-eef6c970e132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:30.537276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb078418-5c46-42b3-b5c2-eef6c970e132 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:30.553667 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 228/472] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:30.559501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb078418-5c46-42b3-b5c2-eef6c970e132 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:30.606611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:30.670909 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a3c5b-1cf6-4b93-b833-c12f9e132115 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:30.708279 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb078418-5c46-42b3-b5c2-eef6c970e132 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:30.713461 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2b9a8-fbdd-4783-a967-6316eb01084d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:30.714214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2b9a8-fbdd-4783-a967-6316eb01084d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:30.714504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2b9a8-fbdd-4783-a967-6316eb01084d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:30.714781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2b9a8-fbdd-4783-a967-6316eb01084d None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:30.715522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2b9a8-fbdd-4783-a967-6316eb01084d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:30.715832 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2b9a8-fbdd-4783-a967-6316eb01084d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:30.718506 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2b9a8-fbdd-4783-a967-6316eb01084d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:30.731390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 245/473] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:30.732736 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe9b33-8bed-4369-82f0-7a9467404fb3 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:30.733062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe9b33-8bed-4369-82f0-7a9467404fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:30.733349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe9b33-8bed-4369-82f0-7a9467404fb3 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:30.735401 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 246/474] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:30.739869 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe9b33-8bed-4369-82f0-7a9467404fb3 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:30.755946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a1dc4-c521-49af-8624-5c77449fc6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:30.756778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a1dc4-c521-49af-8624-5c77449fc6cb None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:30.757104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a1dc4-c521-49af-8624-5c77449fc6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:30.757805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a1dc4-c521-49af-8624-5c77449fc6cb None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:30.758804 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a1dc4-c521-49af-8624-5c77449fc6cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:30.760443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a1dc4-c521-49af-8624-5c77449fc6cb None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:30.854646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe9b33-8bed-4369-82f0-7a9467404fb3 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:30.864668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 229/475] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:02:30 2022] POST /identity/v3/users => generated 248 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:02:30.866865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edee6c8-73ac-49ba-8629-eaf3a62d2055 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:30.867509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edee6c8-73ac-49ba-8629-eaf3a62d2055 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:30.867800 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edee6c8-73ac-49ba-8629-eaf3a62d2055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:30.875901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:30.891438 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-7edee6c8-73ac-49ba-8629-eaf3a62d2055 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:30.892814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-7edee6c8-73ac-49ba-8629-eaf3a62d2055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:30.938808 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:30.966400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634bd13-4b5d-47f8-a573-0a00e786a589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:30.967215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634bd13-4b5d-47f8-a573-0a00e786a589 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:30.967500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634bd13-4b5d-47f8-a573-0a00e786a589 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:30.968529 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634bd13-4b5d-47f8-a573-0a00e786a589 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:30.969465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634bd13-4b5d-47f8-a573-0a00e786a589 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:30.970956 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634bd13-4b5d-47f8-a573-0a00e786a589 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:30.984595 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 230/476] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:31.009064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edee6c8-73ac-49ba-8629-eaf3a62d2055 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:31.094224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 247/477] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:02:30 2022] POST /identity/v3/users => generated 249 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:02:31.144426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edee6c8-73ac-49ba-8629-eaf3a62d2055 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:31.188471 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dc8f3-8e74-40cc-9976-d77ad23faee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:31.189199 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dc8f3-8e74-40cc-9976-d77ad23faee1 None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:31.189489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dc8f3-8e74-40cc-9976-d77ad23faee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:31.189771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dc8f3-8e74-40cc-9976-d77ad23faee1 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:31.190750 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93dc8f3-8e74-40cc-9976-d77ad23faee1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:31.192236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93dc8f3-8e74-40cc-9976-d77ad23faee1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:31.226845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.sql.core [None req-f93dc8f3-8e74-40cc-9976-d77ad23faee1 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 10:02:31.226845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 10:02:31.226845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [parameters: {'id': 'ac681cc5f4ce4111be74027a750b75ad', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 10:02:31.226845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80502) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 10:02:31.246509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-f93dc8f3-8e74-40cc-9976-d77ad23faee1 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 10:02:31.246509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:31.246509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 10:02:31.246509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 10:02:31.246509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 10:02:31.246509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 10:02:31.246509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 10:02:31.246509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 10:02:31.246509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 10:02:31.246509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application conn.query(q) Jun 15 10:02:31.246509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 10:02:31.246509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 10:02:31.246509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 10:02:31.246509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application result.read() Jun 15 10:02:31.246509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return ref.to_dict() Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 10:02:31.247425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application next(self.gen) Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application yield resource Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application next(self.gen) Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise self.value Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application session.commit() Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 10:02:31.248297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self.session.flush() Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application compat.raise_( Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 10:02:31.250258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 10:02:31.251108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application conn.query(q) Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application result.read() Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 10:02:31.251887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [parameters: {'id': 'ac681cc5f4ce4111be74027a750b75ad', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:31.252668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 15 10:02:31.253523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:31.254370 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 231/478] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Jun 15 10:02:31 2022] POST /identity/v3/roles => generated 144 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:31.261542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634bd13-4b5d-47f8-a573-0a00e786a589 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:31.381632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634bd13-4b5d-47f8-a573-0a00e786a589 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:31.423052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ecc848-70a3-4c69-a711-b3ef92a485f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:31.424213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecc848-70a3-4c69-a711-b3ef92a485f6 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:31.424703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecc848-70a3-4c69-a711-b3ef92a485f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:31.425239 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecc848-70a3-4c69-a711-b3ef92a485f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:31.437183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ecc848-70a3-4c69-a711-b3ef92a485f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-f2ecc848-70a3-4c69-a711-b3ef92a485f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:31.446001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:31.446927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:31.447738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:31.447738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:31.447738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:31.447738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:31.447738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:31.447738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:31.447738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:31.447738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:31.447738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:31.447738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:31.447738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 15 10:02:31.447738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:31.453796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 248/479] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:02:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:31.457398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dc8f3-8e74-40cc-9976-d77ad23faee1 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:31.494533 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:31.585747 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dc8f3-8e74-40cc-9976-d77ad23faee1 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:31.632496 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d06ea-fc43-4dcd-b0f2-c3efb6265917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:31.633203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d06ea-fc43-4dcd-b0f2-c3efb6265917 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:31.633490 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d06ea-fc43-4dcd-b0f2-c3efb6265917 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:31.633767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d06ea-fc43-4dcd-b0f2-c3efb6265917 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:31.634533 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d06ea-fc43-4dcd-b0f2-c3efb6265917 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:31.634839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d06ea-fc43-4dcd-b0f2-c3efb6265917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:31.636212 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d06ea-fc43-4dcd-b0f2-c3efb6265917 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:31.646660 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 232/480] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:32.492568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208bfa1-2e46-4f68-af3b-1c0a499db429 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:32.493127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208bfa1-2e46-4f68-af3b-1c0a499db429 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:32.493203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208bfa1-2e46-4f68-af3b-1c0a499db429 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:32.494818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 249/481] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:32.500935 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa2370-1f9a-4ab6-a5a7-474780ba4101 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:32.501425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa2370-1f9a-4ab6-a5a7-474780ba4101 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:32.501638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa2370-1f9a-4ab6-a5a7-474780ba4101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:32.519681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-d8aa2370-1f9a-4ab6-a5a7-474780ba4101 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:32.520892 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d8aa2370-1f9a-4ab6-a5a7-474780ba4101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:32.626973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 233/482] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:02:32.662683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208bfa1-2e46-4f68-af3b-1c0a499db429 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:32.695158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:32.809804 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208bfa1-2e46-4f68-af3b-1c0a499db429 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:32.852542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46503a-b701-46c6-9115-f24fb30ecd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:32.853395 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46503a-b701-46c6-9115-f24fb30ecd8d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:32.853687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46503a-b701-46c6-9115-f24fb30ecd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:32.853970 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46503a-b701-46c6-9115-f24fb30ecd8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:32.870728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46503a-b701-46c6-9115-f24fb30ecd8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-4f46503a-b701-46c6-9115-f24fb30ecd8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:32.880754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:32.882537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:32.882537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:32.882537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:32.882537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:32.882537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:32.882537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:32.882537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:32.882537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:32.882537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:32.882537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:32.882537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:32.882537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:32.883711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 250/483] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:32.891016 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa2370-1f9a-4ab6-a5a7-474780ba4101 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:32.940098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:33.066983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa2370-1f9a-4ab6-a5a7-474780ba4101 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:33.127433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1aef8-11d2-47b2-98c1-b9e9194239fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:33.128121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1aef8-11d2-47b2-98c1-b9e9194239fb None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:33.128402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1aef8-11d2-47b2-98c1-b9e9194239fb None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:33.128676 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1aef8-11d2-47b2-98c1-b9e9194239fb None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:33.129473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1aef8-11d2-47b2-98c1-b9e9194239fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:33.129782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1aef8-11d2-47b2-98c1-b9e9194239fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:33.131158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1aef8-11d2-47b2-98c1-b9e9194239fb None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:33.145885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 234/484] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:33.151701 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46503a-b701-46c6-9115-f24fb30ecd8d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:33.289376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:33.310653 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46503a-b701-46c6-9115-f24fb30ecd8d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:33.357958 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947517e9-c632-4f1d-a206-a2a063649799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:33.359284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947517e9-c632-4f1d-a206-a2a063649799 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:33.359760 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947517e9-c632-4f1d-a206-a2a063649799 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:33.360241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947517e9-c632-4f1d-a206-a2a063649799 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:33.374456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947517e9-c632-4f1d-a206-a2a063649799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:33.376064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947517e9-c632-4f1d-a206-a2a063649799 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:33.397028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-947517e9-c632-4f1d-a206-a2a063649799 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:33.397028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:33.397028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:33.397028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:33.397028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:33.397028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:33.397028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:33.397028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:33.397028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:33.397028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:33.397028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:33.397028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:33.397028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:33.397028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:33.397028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:33.397946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:33.399259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:33.399259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:33.399259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:33.399259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:33.399259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 251/485] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:33.407534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1aef8-11d2-47b2-98c1-b9e9194239fb None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:33.443577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd4212-8ae2-42e9-8f5e-ec3626c57846 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:33.444064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd4212-8ae2-42e9-8f5e-ec3626c57846 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:33.444352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd4212-8ae2-42e9-8f5e-ec3626c57846 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:33.446125 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 252/486] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:02:33.453940 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4064e-ad7e-4199-a0e8-01bbeb662ab7 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:33.454252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4064e-ad7e-4199-a0e8-01bbeb662ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:33.454507 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4064e-ad7e-4199-a0e8-01bbeb662ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:33.494236 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-6db4064e-ad7e-4199-a0e8-01bbeb662ab7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:33.495489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6db4064e-ad7e-4199-a0e8-01bbeb662ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:33.584029 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 253/487] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:02:33.618178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4064e-ad7e-4199-a0e8-01bbeb662ab7 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:33.646569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:33.684545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1aef8-11d2-47b2-98c1-b9e9194239fb None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:33.731549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:33.758315 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46771eb1-63fe-4727-b2d8-b5fd298d2b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:33.759008 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46771eb1-63fe-4727-b2d8-b5fd298d2b53 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:33.759296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46771eb1-63fe-4727-b2d8-b5fd298d2b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:33.759573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46771eb1-63fe-4727-b2d8-b5fd298d2b53 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:33.760325 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46771eb1-63fe-4727-b2d8-b5fd298d2b53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:33.760633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46771eb1-63fe-4727-b2d8-b5fd298d2b53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:33.765695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46771eb1-63fe-4727-b2d8-b5fd298d2b53 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:33.768064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4064e-ad7e-4199-a0e8-01bbeb662ab7 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:33.795542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 235/488] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:33.802124 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46771eb1-63fe-4727-b2d8-b5fd298d2b53 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:33.805495 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e50f8-19f3-4092-aa30-dc717b84bf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:33.806320 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e50f8-19f3-4092-aa30-dc717b84bf1b None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:33.806606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e50f8-19f3-4092-aa30-dc717b84bf1b None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:33.806887 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e50f8-19f3-4092-aa30-dc717b84bf1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:33.818273 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e50f8-19f3-4092-aa30-dc717b84bf1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-eb0e50f8-19f3-4092-aa30-dc717b84bf1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:33.826371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:33.827483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:33.828684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:33.828684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:33.828684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:33.828684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:33.828684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:33.828684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:33.828684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:33.828684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:33.828684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:33.828684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:33.828684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:33.828684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:33.828684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 254/489] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:33.833658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e50f8-19f3-4092-aa30-dc717b84bf1b None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:33.949932 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e50f8-19f3-4092-aa30-dc717b84bf1b None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:33.981374 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:33.994463 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bfec1-8292-4b16-8fc1-68ba57dab66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:33.996063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bfec1-8292-4b16-8fc1-68ba57dab66e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:33.996063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bfec1-8292-4b16-8fc1-68ba57dab66e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:33.996063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bfec1-8292-4b16-8fc1-68ba57dab66e None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:33.996633 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bfec1-8292-4b16-8fc1-68ba57dab66e None admin] RBAC: Adding query filter params (name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:33.996946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bfec1-8292-4b16-8fc1-68ba57dab66e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:33.998397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bfec1-8292-4b16-8fc1-68ba57dab66e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:34.008462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 255/490] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:34.009901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b688bb-e1dc-40da-bbb7-baf29a83f56e None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:34.010072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46771eb1-63fe-4727-b2d8-b5fd298d2b53 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:34.010179 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b688bb-e1dc-40da-bbb7-baf29a83f56e None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:34.010447 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b688bb-e1dc-40da-bbb7-baf29a83f56e None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:34.011933 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 256/491] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:34.014674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75db78f-aa32-4113-9136-50d36170b2a9 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:34.015254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75db78f-aa32-4113-9136-50d36170b2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:34.015639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75db78f-aa32-4113-9136-50d36170b2a9 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:34.018699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 257/492] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:34.019524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75db78f-aa32-4113-9136-50d36170b2a9 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:34.056194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8dabad-3d79-44ed-9446-e1cd1b795373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:34.057138 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dabad-3d79-44ed-9446-e1cd1b795373 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:34.057424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dabad-3d79-44ed-9446-e1cd1b795373 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:34.057755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dabad-3d79-44ed-9446-e1cd1b795373 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:34.066156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8dabad-3d79-44ed-9446-e1cd1b795373 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:34.067606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8dabad-3d79-44ed-9446-e1cd1b795373 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-5a8dabad-3d79-44ed-9446-e1cd1b795373 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:34.084866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:34.085834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:34.086737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:34.086737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:34.086737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:34.086737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:34.086737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:34.086737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:34.086737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:34.086737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:34.086737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:02:34.086737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:34.086737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 236/493] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jun 15 10:02:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:34.087994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83171060-22d1-4645-9c88-86dbe3313e9b None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:34.087994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83171060-22d1-4645-9c88-86dbe3313e9b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:34.087994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83171060-22d1-4645-9c88-86dbe3313e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:34.095842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:34.108159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-83171060-22d1-4645-9c88-86dbe3313e9b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:34.110222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-83171060-22d1-4645-9c88-86dbe3313e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:34.138604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75db78f-aa32-4113-9136-50d36170b2a9 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:34.176984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df67207-4cda-4624-91ce-7c3f393147e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:34.179286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df67207-4cda-4624-91ce-7c3f393147e0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:34.179573 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df67207-4cda-4624-91ce-7c3f393147e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:34.179848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df67207-4cda-4624-91ce-7c3f393147e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:34.191725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df67207-4cda-4624-91ce-7c3f393147e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:34.193238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df67207-4cda-4624-91ce-7c3f393147e0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:34.197985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 237/494] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:34.199360 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fee117-5947-4a43-be2a-582d6eddb1cf None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:34.199656 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fee117-5947-4a43-be2a-582d6eddb1cf None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:34.199994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fee117-5947-4a43-be2a-582d6eddb1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:34.213935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-5df67207-4cda-4624-91ce-7c3f393147e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:34.213935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:34.213935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:34.213935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:34.213935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:34.213935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:34.213935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:34.213935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:34.213935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:34.213935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:34.213935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:34.213935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:34.213935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:34.213935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:34.213935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:34.214806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:34.215634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:34.215634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:34.215634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:34.215634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:34.222706 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 258/495] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:34.223852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df67207-4cda-4624-91ce-7c3f393147e0 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:34.226481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-d6fee117-5947-4a43-be2a-582d6eddb1cf None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:34.227668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d6fee117-5947-4a43-be2a-582d6eddb1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:34.269902 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:34.300280 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 238/496] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:34.310546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fee117-5947-4a43-be2a-582d6eddb1cf None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:34.334803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df67207-4cda-4624-91ce-7c3f393147e0 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:34.370955 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b13485-19b6-4f71-99e1-32998866a68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:34.371630 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b13485-19b6-4f71-99e1-32998866a68e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:34.371921 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b13485-19b6-4f71-99e1-32998866a68e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:34.372199 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b13485-19b6-4f71-99e1-32998866a68e None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:34.373079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b13485-19b6-4f71-99e1-32998866a68e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:34.373395 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b13485-19b6-4f71-99e1-32998866a68e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:34.374919 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b13485-19b6-4f71-99e1-32998866a68e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:34.382764 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:34.414325 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 259/497] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jun 15 10:02:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:34.415159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b13485-19b6-4f71-99e1-32998866a68e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:34.427211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fee117-5947-4a43-be2a-582d6eddb1cf None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:34.467720 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa42ff-4ab7-49d0-9ee5-4cc5bcda985b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:34.468389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa42ff-4ab7-49d0-9ee5-4cc5bcda985b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:34.468664 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa42ff-4ab7-49d0-9ee5-4cc5bcda985b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:34.468948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa42ff-4ab7-49d0-9ee5-4cc5bcda985b None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:34.469717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa42ff-4ab7-49d0-9ee5-4cc5bcda985b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:34.470031 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa42ff-4ab7-49d0-9ee5-4cc5bcda985b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:34.471425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa42ff-4ab7-49d0-9ee5-4cc5bcda985b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:34.488158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 239/498] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:34.488158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa42ff-4ab7-49d0-9ee5-4cc5bcda985b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:34.529670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b13485-19b6-4f71-99e1-32998866a68e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:34.532798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:34.570468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e11869-95b0-425a-9b73-06a69e42d23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:34.571668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e11869-95b0-425a-9b73-06a69e42d23d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:34.572178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e11869-95b0-425a-9b73-06a69e42d23d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:34.572657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e11869-95b0-425a-9b73-06a69e42d23d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:34.586331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e11869-95b0-425a-9b73-06a69e42d23d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-91e11869-95b0-425a-9b73-06a69e42d23d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:34.595489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:34.596465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:34.597348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:34.597348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:34.597348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:34.597348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:34.597348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:34.597348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:34.597348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:34.597348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:34.597348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:34.597348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:34.597348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:34.597348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:34.603442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 260/499] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:34.603442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e11869-95b0-425a-9b73-06a69e42d23d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:34.610672 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa42ff-4ab7-49d0-9ee5-4cc5bcda985b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:34.636185 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:34.648271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6a074-189e-466d-9d89-97c2b630ca56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:34.649418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6a074-189e-466d-9d89-97c2b630ca56 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:34.649418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6a074-189e-466d-9d89-97c2b630ca56 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:34.649698 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6a074-189e-466d-9d89-97c2b630ca56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:34.661452 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6a074-189e-466d-9d89-97c2b630ca56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-3ed6a074-189e-466d-9d89-97c2b630ca56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:34.669653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:34.670739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:34.672046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:34.672046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:34.672046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:34.672046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:34.672046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:34.672046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:34.672046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:34.672046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:34.672046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:34.672046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:34.672046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:34.672046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:34.676734 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 240/500] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:34.677670 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6a074-189e-466d-9d89-97c2b630ca56 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:34.708976 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e11869-95b0-425a-9b73-06a69e42d23d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:34.742773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be7365-c426-4f3b-b0b1-d6ba04a77f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:34.743424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be7365-c426-4f3b-b0b1-d6ba04a77f47 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:34.743708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be7365-c426-4f3b-b0b1-d6ba04a77f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:34.743981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be7365-c426-4f3b-b0b1-d6ba04a77f47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:34.772561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:34.777186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6a074-189e-466d-9d89-97c2b630ca56 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:34.813254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0962220-dbd8-454f-9df0-fb857e512cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:34.814060 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0962220-dbd8-454f-9df0-fb857e512cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:34.814351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0962220-dbd8-454f-9df0-fb857e512cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:34.814632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0962220-dbd8-454f-9df0-fb857e512cd6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:34.815678 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be7365-c426-4f3b-b0b1-d6ba04a77f47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:34.817156 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be7365-c426-4f3b-b0b1-d6ba04a77f47 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:34.822076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0962220-dbd8-454f-9df0-fb857e512cd6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:34.823535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0962220-dbd8-454f-9df0-fb857e512cd6 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-d0962220-dbd8-454f-9df0-fb857e512cd6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:34.838431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:34.839392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:34.840260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:34.840260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:34.840260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:34.840260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:34.840260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:34.840260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:34.840260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:34.840260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:34.840260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 10:02:34.840260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:34.840260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 241/501] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:34.840816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0962220-dbd8-454f-9df0-fb857e512cd6 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:34.890944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:34.920457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 261/502] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:34.922248 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a305a9-307b-4190-a409-e1b492969abc None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:34.922539 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a305a9-307b-4190-a409-e1b492969abc None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:34.922819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a305a9-307b-4190-a409-e1b492969abc None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:34.924251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 262/503] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:34.924788 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a305a9-307b-4190-a409-e1b492969abc None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:34.949819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0962220-dbd8-454f-9df0-fb857e512cd6 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:34.984133 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ac436-a196-4933-82b3-6a6e7205c147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:34.984794 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ac436-a196-4933-82b3-6a6e7205c147 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:34.985113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ac436-a196-4933-82b3-6a6e7205c147 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:34.985390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ac436-a196-4933-82b3-6a6e7205c147 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:34.986150 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ac436-a196-4933-82b3-6a6e7205c147 None admin] RBAC: Adding query filter params (name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:34.986461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ac436-a196-4933-82b3-6a6e7205c147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:34.987853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ac436-a196-4933-82b3-6a6e7205c147 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:34.995866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 242/504] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:34.996814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ac436-a196-4933-82b3-6a6e7205c147 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:34.997413 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:35.040619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a305a9-307b-4190-a409-e1b492969abc None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:35.084641 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d9643-3df1-4295-bb1d-2cb1e65a202e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:35.085340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d9643-3df1-4295-bb1d-2cb1e65a202e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:35.085624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d9643-3df1-4295-bb1d-2cb1e65a202e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:35.085897 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d9643-3df1-4295-bb1d-2cb1e65a202e None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:35.086643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d9643-3df1-4295-bb1d-2cb1e65a202e None admin] RBAC: Adding query filter params (name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:35.086949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d9643-3df1-4295-bb1d-2cb1e65a202e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:35.088330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d9643-3df1-4295-bb1d-2cb1e65a202e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:35.096853 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 263/505] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:35.097700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d9643-3df1-4295-bb1d-2cb1e65a202e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:35.102219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ac436-a196-4933-82b3-6a6e7205c147 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:35.109144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:35.136172 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7504a-9134-4c97-bb8c-ece3db3fe9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:35.136842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7504a-9134-4c97-bb8c-ece3db3fe9cf None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:35.137156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7504a-9134-4c97-bb8c-ece3db3fe9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:35.137444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7504a-9134-4c97-bb8c-ece3db3fe9cf None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:35.138294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7504a-9134-4c97-bb8c-ece3db3fe9cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:35.138625 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7504a-9134-4c97-bb8c-ece3db3fe9cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:35.140938 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7504a-9134-4c97-bb8c-ece3db3fe9cf None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:35.178883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 243/506] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Jun 15 10:02:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:35.180245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4a4e0-643d-4aca-aa94-529deaf985b6 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:35.180545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4a4e0-643d-4aca-aa94-529deaf985b6 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:35.180830 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4a4e0-643d-4aca-aa94-529deaf985b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:35.197529 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a7a4a4e0-643d-4aca-aa94-529deaf985b6 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:35.198740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a7a4a4e0-643d-4aca-aa94-529deaf985b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:35.204728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d9643-3df1-4295-bb1d-2cb1e65a202e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:35.227908 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:35.239608 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91047f7d-c8b3-4259-bad3-c9c932a1f94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:35.244618 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91047f7d-c8b3-4259-bad3-c9c932a1f94c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:35.244618 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91047f7d-c8b3-4259-bad3-c9c932a1f94c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:35.244618 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91047f7d-c8b3-4259-bad3-c9c932a1f94c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:35.257961 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91047f7d-c8b3-4259-bad3-c9c932a1f94c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:35.259344 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91047f7d-c8b3-4259-bad3-c9c932a1f94c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:35.270614 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 244/507] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:35.271541 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4a4e0-643d-4aca-aa94-529deaf985b6 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:35.281142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-91047f7d-c8b3-4259-bad3-c9c932a1f94c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:35.281142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:35.281142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:35.281142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:35.281142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:35.281142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:35.281142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:35.281142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:35.281142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:35.281142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:35.281142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:35.281142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:35.281142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:35.281142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:35.281142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:35.289576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:35.294244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:35.294244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:35.294244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:35.294244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:35.294244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 264/508] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:35.294244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91047f7d-c8b3-4259-bad3-c9c932a1f94c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:35.373234 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:35.379968 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4a4e0-643d-4aca-aa94-529deaf985b6 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:35.395769 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91047f7d-c8b3-4259-bad3-c9c932a1f94c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:35.414177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4becc-a573-4a4e-985b-582f6891ae20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:35.414978 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4becc-a573-4a4e-985b-582f6891ae20 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:35.415265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4becc-a573-4a4e-985b-582f6891ae20 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:35.415544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4becc-a573-4a4e-985b-582f6891ae20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:35.427328 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4becc-a573-4a4e-985b-582f6891ae20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:35.428827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4becc-a573-4a4e-985b-582f6891ae20 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:35.430998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3458a7-9a65-4990-85f4-ab4ddc47b2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:35.431803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3458a7-9a65-4990-85f4-ab4ddc47b2de None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:35.432095 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3458a7-9a65-4990-85f4-ab4ddc47b2de None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:35.432370 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3458a7-9a65-4990-85f4-ab4ddc47b2de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:35.443463 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3458a7-9a65-4990-85f4-ab4ddc47b2de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:35.444982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3458a7-9a65-4990-85f4-ab4ddc47b2de None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:35.448064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-b8f4becc-a573-4a4e-985b-582f6891ae20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:35.448064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:35.448064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:35.448064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:35.448064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:35.448064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:35.448064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:35.448064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:35.448064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:35.448064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:35.448064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:35.448064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:35.448064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:35.448064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:35.448064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:35.448989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:35.449871 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:35.449871 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:35.449871 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:35.449871 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:35.449871 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 245/509] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:35.450138 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4becc-a573-4a4e-985b-582f6891ae20 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:35.464805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-fc3458a7-9a65-4990-85f4-ab4ddc47b2de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:35.464805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:35.464805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:35.464805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:35.464805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:35.464805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:35.464805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:35.464805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:35.464805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:35.464805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:35.464805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:35.464805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:35.464805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:35.464805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:35.464805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:35.465884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:35.467039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:35.467039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:35.467039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:35.467039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:35.467039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 265/510] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:35.467423 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3458a7-9a65-4990-85f4-ab4ddc47b2de None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:35.496562 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:35.558141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4becc-a573-4a4e-985b-582f6891ae20 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:35.574670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3458a7-9a65-4990-85f4-ab4ddc47b2de None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:35.594716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdabd23-f923-46ee-ad36-1315206cee08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:35.595359 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdabd23-f923-46ee-ad36-1315206cee08 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:35.595643 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdabd23-f923-46ee-ad36-1315206cee08 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:35.595917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdabd23-f923-46ee-ad36-1315206cee08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:35.609572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c936c40-3469-4b58-a61a-0e28168377a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:35.610230 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c936c40-3469-4b58-a61a-0e28168377a5 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:35.610519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c936c40-3469-4b58-a61a-0e28168377a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:35.610798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c936c40-3469-4b58-a61a-0e28168377a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:35.611527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c936c40-3469-4b58-a61a-0e28168377a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:35.611843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c936c40-3469-4b58-a61a-0e28168377a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:35.611995 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:35.613258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c936c40-3469-4b58-a61a-0e28168377a5 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:35.625118 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 266/511] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:35.625950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c936c40-3469-4b58-a61a-0e28168377a5 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:35.665792 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdabd23-f923-46ee-ad36-1315206cee08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:35.667317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdabd23-f923-46ee-ad36-1315206cee08 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:35.709397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:35.724689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c936c40-3469-4b58-a61a-0e28168377a5 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:35.758932 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe38e5b-8b17-4177-8a71-a0b6555cff8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:35.759828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe38e5b-8b17-4177-8a71-a0b6555cff8d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:35.760114 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe38e5b-8b17-4177-8a71-a0b6555cff8d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:35.760397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe38e5b-8b17-4177-8a71-a0b6555cff8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:35.766872 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 246/512] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:35.767519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdabd23-f923-46ee-ad36-1315206cee08 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:35.771458 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe38e5b-8b17-4177-8a71-a0b6555cff8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:35.773022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe38e5b-8b17-4177-8a71-a0b6555cff8d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:35.790270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-4fe38e5b-8b17-4177-8a71-a0b6555cff8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:35.790270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:35.790270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:35.790270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:35.790270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:35.790270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:35.790270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:35.790270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:35.790270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:35.790270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:35.790270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:35.790270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:35.790270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:35.790270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:35.790270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:35.791216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:35.792469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:35.792469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:35.792469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:35.792469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:35.792469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 267/513] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:35.792857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe38e5b-8b17-4177-8a71-a0b6555cff8d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:35.820710 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:35.874454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdabd23-f923-46ee-ad36-1315206cee08 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:35.891437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe38e5b-8b17-4177-8a71-a0b6555cff8d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:35.907735 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e405a6b-82fb-4fb1-b631-1783e644b9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:35.908390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405a6b-82fb-4fb1-b631-1783e644b9bb None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:35.908669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405a6b-82fb-4fb1-b631-1783e644b9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:35.908939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e405a6b-82fb-4fb1-b631-1783e644b9bb None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:35.909691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e405a6b-82fb-4fb1-b631-1783e644b9bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:35.909995 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e405a6b-82fb-4fb1-b631-1783e644b9bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:35.911386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e405a6b-82fb-4fb1-b631-1783e644b9bb None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:35.922870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 247/514] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:35.923664 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e405a6b-82fb-4fb1-b631-1783e644b9bb None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:35.929554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550fdf6-eeea-42c7-a35f-38488afee96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:35.930221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550fdf6-eeea-42c7-a35f-38488afee96d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:35.930503 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550fdf6-eeea-42c7-a35f-38488afee96d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:35.930780 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550fdf6-eeea-42c7-a35f-38488afee96d None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:35.931513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550fdf6-eeea-42c7-a35f-38488afee96d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:35.931820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550fdf6-eeea-42c7-a35f-38488afee96d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:35.933281 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550fdf6-eeea-42c7-a35f-38488afee96d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:35.941093 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:35.946211 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 268/515] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:35.947003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550fdf6-eeea-42c7-a35f-38488afee96d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:36.031145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e405a6b-82fb-4fb1-b631-1783e644b9bb None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:36.038050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:36.044566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550fdf6-eeea-42c7-a35f-38488afee96d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:36.066706 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5436b0-6cb7-4df2-8241-02af03314f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:36.067496 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5436b0-6cb7-4df2-8241-02af03314f97 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:36.067785 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5436b0-6cb7-4df2-8241-02af03314f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:36.068062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5436b0-6cb7-4df2-8241-02af03314f97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:36.074783 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5436b0-6cb7-4df2-8241-02af03314f97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:36.076238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5436b0-6cb7-4df2-8241-02af03314f97 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:36.078855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32bbe5-2352-476a-a536-a90d10e54a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:36.079651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32bbe5-2352-476a-a536-a90d10e54a5c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:36.079930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32bbe5-2352-476a-a536-a90d10e54a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:36.080202 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32bbe5-2352-476a-a536-a90d10e54a5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:36.090138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32bbe5-2352-476a-a536-a90d10e54a5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:36.091059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-0e5436b0-6cb7-4df2-8241-02af03314f97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:36.091059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:36.091059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:36.091059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:36.091059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:36.091059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:36.091059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:36.091059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:36.091059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:36.091059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:36.091059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:36.091059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:36.091059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:36.091059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:36.091059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:36.091974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:36.093546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:36.093546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:36.093546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:36.093546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:36.093546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:36.093546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:36.093546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:36.093546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:36.093955 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32bbe5-2352-476a-a536-a90d10e54a5c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:36.094020 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 248/516] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:36.094020 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5436b0-6cb7-4df2-8241-02af03314f97 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:36.116550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-4e32bbe5-2352-476a-a536-a90d10e54a5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:36.116550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:36.116550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:36.116550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:36.116550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:36.116550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:36.116550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:36.116550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:36.116550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:36.116550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:36.116550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:36.116550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:36.116550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:36.116550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:36.116550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:36.117468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:36.118297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:36.118297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:36.118297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:36.118297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:36.118297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 269/517] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:36.118604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32bbe5-2352-476a-a536-a90d10e54a5c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:36.169207 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:36.203818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5436b0-6cb7-4df2-8241-02af03314f97 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:36.222657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32bbe5-2352-476a-a536-a90d10e54a5c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:36.238702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5c998-daf2-4a40-b094-dfa53b0e2094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:36.239362 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5c998-daf2-4a40-b094-dfa53b0e2094 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:36.239648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5c998-daf2-4a40-b094-dfa53b0e2094 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:36.239927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5c998-daf2-4a40-b094-dfa53b0e2094 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:36.240659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5c998-daf2-4a40-b094-dfa53b0e2094 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:36.240984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5c998-daf2-4a40-b094-dfa53b0e2094 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:36.242373 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5c998-daf2-4a40-b094-dfa53b0e2094 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:36.253524 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 249/518] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:36.254332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5c998-daf2-4a40-b094-dfa53b0e2094 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:36.257159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e4608-9c53-4952-a3b4-b505d59e439a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:36.258173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e4608-9c53-4952-a3b4-b505d59e439a None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:36.258306 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e4608-9c53-4952-a3b4-b505d59e439a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:36.258561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e4608-9c53-4952-a3b4-b505d59e439a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:36.265936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e4608-9c53-4952-a3b4-b505d59e439a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:36.267394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e4608-9c53-4952-a3b4-b505d59e439a None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:36.282798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-d07e4608-9c53-4952-a3b4-b505d59e439a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:36.284138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:36.285048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:36.285962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:36.285962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:36.285962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:36.285962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:36.285962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:36.285962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:36.285962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:36.285962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:36.285962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:02:36.285962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:36.285962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 270/519] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:36.286505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e4608-9c53-4952-a3b4-b505d59e439a None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:36.361327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5c998-daf2-4a40-b094-dfa53b0e2094 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:36.386758 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e4608-9c53-4952-a3b4-b505d59e439a None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:36.394820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4826b-ba65-4a2a-a1c2-e1730c460407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:36.395619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4826b-ba65-4a2a-a1c2-e1730c460407 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:36.395899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4826b-ba65-4a2a-a1c2-e1730c460407 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:36.396175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4826b-ba65-4a2a-a1c2-e1730c460407 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:36.398210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:36.404255 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4826b-ba65-4a2a-a1c2-e1730c460407 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:36.406451 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4826b-ba65-4a2a-a1c2-e1730c460407 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-57e4826b-ba65-4a2a-a1c2-e1730c460407 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:36.420646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:36.421903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:36.422779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:36.422779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:36.422779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:36.422779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:36.422779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:36.422779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:36.422779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:36.422779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:36.422779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 10:02:36.422779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:36.422779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 250/520] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:36.423328 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4826b-ba65-4a2a-a1c2-e1730c460407 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:36.424363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded0666-25cf-49e5-9e6a-f263170eec96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:36.425061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded0666-25cf-49e5-9e6a-f263170eec96 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:36.425345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded0666-25cf-49e5-9e6a-f263170eec96 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:36.425877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded0666-25cf-49e5-9e6a-f263170eec96 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:36.426679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded0666-25cf-49e5-9e6a-f263170eec96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:36.428048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded0666-25cf-49e5-9e6a-f263170eec96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:36.428528 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded0666-25cf-49e5-9e6a-f263170eec96 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:36.446564 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 271/521] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:36.447494 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded0666-25cf-49e5-9e6a-f263170eec96 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:36.508887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:36.523962 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4826b-ba65-4a2a-a1c2-e1730c460407 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:36.551494 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded0666-25cf-49e5-9e6a-f263170eec96 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:36.560579 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824db72f-881e-41e9-ab6b-f53ba159d2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:36.561309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824db72f-881e-41e9-ab6b-f53ba159d2be None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:36.561734 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824db72f-881e-41e9-ab6b-f53ba159d2be None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:36.562021 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824db72f-881e-41e9-ab6b-f53ba159d2be None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:36.562755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824db72f-881e-41e9-ab6b-f53ba159d2be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:36.563062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824db72f-881e-41e9-ab6b-f53ba159d2be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:36.564474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824db72f-881e-41e9-ab6b-f53ba159d2be None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:36.576065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 251/522] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:36.577603 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824db72f-881e-41e9-ab6b-f53ba159d2be None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:36.586964 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89132df9-687d-4114-92ba-319b4b492ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:36.587642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89132df9-687d-4114-92ba-319b4b492ac0 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:36.587932 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89132df9-687d-4114-92ba-319b4b492ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:36.588217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89132df9-687d-4114-92ba-319b4b492ac0 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:36.588926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89132df9-687d-4114-92ba-319b4b492ac0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:36.590399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89132df9-687d-4114-92ba-319b4b492ac0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:36.675155 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824db72f-881e-41e9-ab6b-f53ba159d2be None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:36.683447 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 272/523] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:02:36 2022] POST /identity/v3/users => generated 247 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:02:36.684280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89132df9-687d-4114-92ba-319b4b492ac0 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:36.714681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdf16b-b77f-462b-8a9d-32149ec3788e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:36.715366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdf16b-b77f-462b-8a9d-32149ec3788e None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:36.715739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdf16b-b77f-462b-8a9d-32149ec3788e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:36.716013 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdf16b-b77f-462b-8a9d-32149ec3788e None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:36.716881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdf16b-b77f-462b-8a9d-32149ec3788e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:36.717226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdf16b-b77f-462b-8a9d-32149ec3788e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:36.719941 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdf16b-b77f-462b-8a9d-32149ec3788e None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:36.729793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:36.746005 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 252/524] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Jun 15 10:02:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:36.746005 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdf16b-b77f-462b-8a9d-32149ec3788e None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:36.750917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:36.810484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89132df9-687d-4114-92ba-319b4b492ac0 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:36.853354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72570e6f-6a9f-4231-b69d-ec32a83d7458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:36.854462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72570e6f-6a9f-4231-b69d-ec32a83d7458 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:36.854754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72570e6f-6a9f-4231-b69d-ec32a83d7458 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:36.856192 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72570e6f-6a9f-4231-b69d-ec32a83d7458 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:36.856192 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72570e6f-6a9f-4231-b69d-ec32a83d7458 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:36.856192 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72570e6f-6a9f-4231-b69d-ec32a83d7458 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:36.857818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72570e6f-6a9f-4231-b69d-ec32a83d7458 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:36.860696 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdf16b-b77f-462b-8a9d-32149ec3788e None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:36.876875 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 273/525] 10.222.0.60 () {56 vars in 1139 bytes} [Wed Jun 15 10:02:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:36.877810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72570e6f-6a9f-4231-b69d-ec32a83d7458 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:36.908279 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6ee35-3ea3-43fe-9190-064aa8b9f397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:36.909069 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6ee35-3ea3-43fe-9190-064aa8b9f397 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:36.909357 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6ee35-3ea3-43fe-9190-064aa8b9f397 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:36.909683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be6ee35-3ea3-43fe-9190-064aa8b9f397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:36.910555 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6ee35-3ea3-43fe-9190-064aa8b9f397 None admin] RBAC: Adding query filter params (role.id=2c276d442f7f4dd59ae7fc3a03943716, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=26f23f8595fc43ad94aee7d0192164a1) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:36.910867 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6ee35-3ea3-43fe-9190-064aa8b9f397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:36.912346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be6ee35-3ea3-43fe-9190-064aa8b9f397 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:36.918962 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 253/526] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:02:36 2022] GET /identity/v3/role_assignments?user.id=26f23f8595fc43ad94aee7d0192164a1&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=2c276d442f7f4dd59ae7fc3a03943716 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:36.920101 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6ee35-3ea3-43fe-9190-064aa8b9f397 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:36.979702 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:37.003865 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72570e6f-6a9f-4231-b69d-ec32a83d7458 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:37.040509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:37.053441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b941136-7291-4ae6-a269-bb71dabc22b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:37.054437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b941136-7291-4ae6-a269-bb71dabc22b0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:37.054818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b941136-7291-4ae6-a269-bb71dabc22b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:37.055299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b941136-7291-4ae6-a269-bb71dabc22b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:37.064201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b941136-7291-4ae6-a269-bb71dabc22b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:37.066233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b941136-7291-4ae6-a269-bb71dabc22b0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:37.079471 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be6ee35-3ea3-43fe-9190-064aa8b9f397 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:37.084091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-5b941136-7291-4ae6-a269-bb71dabc22b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:37.084091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:37.084091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:37.084091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:37.084091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:37.084091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:37.084091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:37.084091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:37.084091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:37.084091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:37.084091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:37.084091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:37.084091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:37.084091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:37.084091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:37.085185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:37.086210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:37.086210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:37.086210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:37.086210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:37.086210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:37.086210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:37.086210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:37.086210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:37.088228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 274/527] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:37.089577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b941136-7291-4ae6-a269-bb71dabc22b0 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:37.142157 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59588aae-ac1f-4419-aab0-c48ea196bd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:37.143826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59588aae-ac1f-4419-aab0-c48ea196bd59 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:37.144483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59588aae-ac1f-4419-aab0-c48ea196bd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:37.145565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59588aae-ac1f-4419-aab0-c48ea196bd59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:37.239173 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59588aae-ac1f-4419-aab0-c48ea196bd59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:37.240860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59588aae-ac1f-4419-aab0-c48ea196bd59 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:37.246741 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b941136-7291-4ae6-a269-bb71dabc22b0 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:37.291684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27413c5-6e49-4cf2-bf12-9cef013444c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:37.292655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27413c5-6e49-4cf2-bf12-9cef013444c6 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:37.293161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27413c5-6e49-4cf2-bf12-9cef013444c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:37.293629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27413c5-6e49-4cf2-bf12-9cef013444c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:37.316002 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:37.333034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:37.381903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 254/528] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:37.383069 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59588aae-ac1f-4419-aab0-c48ea196bd59 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:37.389902 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27413c5-6e49-4cf2-bf12-9cef013444c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:37.391364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27413c5-6e49-4cf2-bf12-9cef013444c6 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:37.496723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59588aae-ac1f-4419-aab0-c48ea196bd59 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:37.524391 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 275/529] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:37.524391 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27413c5-6e49-4cf2-bf12-9cef013444c6 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:37.550220 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71be82-0448-4974-9dda-551aef337312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:37.551412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71be82-0448-4974-9dda-551aef337312 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:37.552230 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71be82-0448-4974-9dda-551aef337312 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:37.552741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71be82-0448-4974-9dda-551aef337312 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:37.553759 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71be82-0448-4974-9dda-551aef337312 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:37.554277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71be82-0448-4974-9dda-551aef337312 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:37.556105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71be82-0448-4974-9dda-551aef337312 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:37.576580 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:37.579824 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 255/530] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:37.580973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71be82-0448-4974-9dda-551aef337312 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:37.615184 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:37.651542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27413c5-6e49-4cf2-bf12-9cef013444c6 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:37.700336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71be82-0448-4974-9dda-551aef337312 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:37.702868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f1d63-8e89-4023-ab12-ff3f5b061e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:37.703997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f1d63-8e89-4023-ab12-ff3f5b061e71 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:37.704290 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f1d63-8e89-4023-ab12-ff3f5b061e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:37.704571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f1d63-8e89-4023-ab12-ff3f5b061e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:37.720129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f1d63-8e89-4023-ab12-ff3f5b061e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:37.721884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f1d63-8e89-4023-ab12-ff3f5b061e71 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:37.746071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-4e0f1d63-8e89-4023-ab12-ff3f5b061e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:37.746071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:37.746071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:37.746071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:37.746071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:37.746071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:37.746071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:37.746071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:37.746071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:37.746071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:37.746071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:37.746071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:37.746071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:37.746071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:37.746071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:37.747016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:37.747850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:37.747850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:37.747850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:37.747850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:37.748072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45c87a-5306-4bc1-90ef-056391b32294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:37.748072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45c87a-5306-4bc1-90ef-056391b32294 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:37.748072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45c87a-5306-4bc1-90ef-056391b32294 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:37.748266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45c87a-5306-4bc1-90ef-056391b32294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:37.754448 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 276/531] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:37.755622 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f1d63-8e89-4023-ab12-ff3f5b061e71 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:37.765933 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45c87a-5306-4bc1-90ef-056391b32294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:37.767478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45c87a-5306-4bc1-90ef-056391b32294 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:37.792767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-3c45c87a-5306-4bc1-90ef-056391b32294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:37.792767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:37.792767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:37.792767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:37.792767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:37.792767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:37.792767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:37.792767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:37.792767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:37.792767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:37.792767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:37.792767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:37.792767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:37.792767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:37.792767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:37.793678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:37.794496 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:37.794496 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:37.794496 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:37.794496 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:37.799985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 256/532] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:37.801577 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45c87a-5306-4bc1-90ef-056391b32294 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:37.880826 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:37.890029 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f1d63-8e89-4023-ab12-ff3f5b061e71 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:37.907716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:37.930127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45c87a-5306-4bc1-90ef-056391b32294 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:37.933909 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f35314-f201-40dd-93d2-ee0ded713afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:37.934659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f35314-f201-40dd-93d2-ee0ded713afc None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:37.935022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f35314-f201-40dd-93d2-ee0ded713afc None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:37.935375 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f35314-f201-40dd-93d2-ee0ded713afc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:37.936909 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f35314-f201-40dd-93d2-ee0ded713afc None admin] RBAC: Adding query filter params (role.id=2c276d442f7f4dd59ae7fc3a03943716, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=7fbf33189aa94f4f973d558d615a23e6) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:37.937784 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f35314-f201-40dd-93d2-ee0ded713afc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:37.938938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f35314-f201-40dd-93d2-ee0ded713afc None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:37.946893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 277/533] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:02:37 2022] GET /identity/v3/role_assignments?user.id=7fbf33189aa94f4f973d558d615a23e6&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=2c276d442f7f4dd59ae7fc3a03943716 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:37.947826 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f35314-f201-40dd-93d2-ee0ded713afc None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:37.974788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6232e-ccad-445e-950d-da8d207403e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:37.975835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6232e-ccad-445e-950d-da8d207403e7 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:37.976364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6232e-ccad-445e-950d-da8d207403e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:37.976841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6232e-ccad-445e-950d-da8d207403e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:37.977811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6232e-ccad-445e-950d-da8d207403e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:37.978376 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6232e-ccad-445e-950d-da8d207403e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:37.980012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6232e-ccad-445e-950d-da8d207403e7 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:37.995708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 257/534] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:38.002036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6232e-ccad-445e-950d-da8d207403e7 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:38.102544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f35314-f201-40dd-93d2-ee0ded713afc None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:38.126727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6232e-ccad-445e-950d-da8d207403e7 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:38.163597 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf7e49-6b85-4d1f-bba2-ddc0592c39a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:38.164878 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf7e49-6b85-4d1f-bba2-ddc0592c39a9 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:38.165437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf7e49-6b85-4d1f-bba2-ddc0592c39a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:38.165960 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf7e49-6b85-4d1f-bba2-ddc0592c39a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:38.167214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf7e49-6b85-4d1f-bba2-ddc0592c39a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:38.167721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf7e49-6b85-4d1f-bba2-ddc0592c39a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:38.167848 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df55ced-8d47-4151-859f-f86e4aa3412d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:38.168730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df55ced-8d47-4151-859f-f86e4aa3412d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:38.169106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df55ced-8d47-4151-859f-f86e4aa3412d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:38.177177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df55ced-8d47-4151-859f-f86e4aa3412d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:38.177349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf7e49-6b85-4d1f-bba2-ddc0592c39a9 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:38.190882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df55ced-8d47-4151-859f-f86e4aa3412d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:38.192382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df55ced-8d47-4151-859f-f86e4aa3412d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:38.201057 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 278/535] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:38.202197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:38.210039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf7e49-6b85-4d1f-bba2-ddc0592c39a9 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:38.222406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-6df55ced-8d47-4151-859f-f86e4aa3412d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:38.222406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:38.222406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:38.222406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:38.222406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:38.222406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:38.222406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:38.222406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:38.222406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:38.222406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:38.222406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:38.222406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:38.222406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:38.222406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:38.222406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:38.223351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:38.224183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:38.224183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:38.224183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:38.224183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:38.224183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:38.224183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:38.224183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:38.224183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:38.226809 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 258/536] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:38.232519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df55ced-8d47-4151-859f-f86e4aa3412d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:38.263732 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:38.412692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df55ced-8d47-4151-859f-f86e4aa3412d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:38.424256 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf7e49-6b85-4d1f-bba2-ddc0592c39a9 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:38.468380 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895a7bc-e880-48c7-8f05-593f53ba6e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:38.469629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895a7bc-e880-48c7-8f05-593f53ba6e82 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:38.470239 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895a7bc-e880-48c7-8f05-593f53ba6e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:38.470722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895a7bc-e880-48c7-8f05-593f53ba6e82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:38.478829 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895a7bc-e880-48c7-8f05-593f53ba6e82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:38.480572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895a7bc-e880-48c7-8f05-593f53ba6e82 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:38.497982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-3895a7bc-e880-48c7-8f05-593f53ba6e82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:38.497982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:38.497982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:38.497982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:38.497982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:38.497982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:38.497982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:38.497982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:38.497982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:38.497982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:38.497982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:38.497982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:38.497982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:38.497982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:38.497982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:38.499421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:38.500259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:38.500259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:38.500259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:38.500259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:38.500259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:38.500259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:38.500259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:38.500259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:38.501864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7b0d0-a229-47dd-a0c5-b667ade351b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:38.502539 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7b0d0-a229-47dd-a0c5-b667ade351b8 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:38.502824 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7b0d0-a229-47dd-a0c5-b667ade351b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:38.503101 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7b0d0-a229-47dd-a0c5-b667ade351b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:38.503710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 279/537] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:38.503901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7b0d0-a229-47dd-a0c5-b667ade351b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:38.508516 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7b0d0-a229-47dd-a0c5-b667ade351b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:38.508516 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7b0d0-a229-47dd-a0c5-b667ade351b8 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:38.509891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895a7bc-e880-48c7-8f05-593f53ba6e82 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:38.536483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 259/538] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:38.542418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7b0d0-a229-47dd-a0c5-b667ade351b8 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:38.630737 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:38.679688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895a7bc-e880-48c7-8f05-593f53ba6e82 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:38.685362 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:38.738300 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7b0d0-a229-47dd-a0c5-b667ade351b8 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:38.775947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a311535-eb1d-48e4-99e0-2b94723f1fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:38.776618 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a311535-eb1d-48e4-99e0-2b94723f1fbd None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:38.776888 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a311535-eb1d-48e4-99e0-2b94723f1fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:38.777189 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a311535-eb1d-48e4-99e0-2b94723f1fbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:38.778044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a311535-eb1d-48e4-99e0-2b94723f1fbd None admin] RBAC: Adding query filter params (role.id=2c276d442f7f4dd59ae7fc3a03943716, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=fa51162a69d44127bd34df2f4653dc15) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:38.778357 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a311535-eb1d-48e4-99e0-2b94723f1fbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:38.779818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a311535-eb1d-48e4-99e0-2b94723f1fbd None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:38.783136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc802c38-13d5-4201-a84c-3287e3b77e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:38.783814 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc802c38-13d5-4201-a84c-3287e3b77e30 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:38.784096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc802c38-13d5-4201-a84c-3287e3b77e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:38.784370 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc802c38-13d5-4201-a84c-3287e3b77e30 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:38.785205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc802c38-13d5-4201-a84c-3287e3b77e30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:38.785514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc802c38-13d5-4201-a84c-3287e3b77e30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:38.787042 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc802c38-13d5-4201-a84c-3287e3b77e30 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:38.799659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 260/539] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:02:38 2022] GET /identity/v3/role_assignments?user.id=fa51162a69d44127bd34df2f4653dc15&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=2c276d442f7f4dd59ae7fc3a03943716 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:38.813187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 280/540] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:38.819788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a311535-eb1d-48e4-99e0-2b94723f1fbd None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:38.941833 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a311535-eb1d-48e4-99e0-2b94723f1fbd None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:38.968643 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:38.991664 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d2109-14fc-4eaa-8be7-08d331903ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:38.992615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d2109-14fc-4eaa-8be7-08d331903ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:38.992900 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d2109-14fc-4eaa-8be7-08d331903ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:38.993202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d2109-14fc-4eaa-8be7-08d331903ef0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:38.994072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d2109-14fc-4eaa-8be7-08d331903ef0 None admin] RBAC: Adding query filter params (role.id=2c276d442f7f4dd59ae7fc3a03943716, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=ae926ce83d464c768baddd90de31aa40) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:38.994384 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d2109-14fc-4eaa-8be7-08d331903ef0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:38.995903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d2109-14fc-4eaa-8be7-08d331903ef0 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:39.007626 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 261/541] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:02:38 2022] GET /identity/v3/role_assignments?user.id=ae926ce83d464c768baddd90de31aa40&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=2c276d442f7f4dd59ae7fc3a03943716 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:39.133465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c3b44-7d1f-440d-bbce-84f32dd4d6d3 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:39.133811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c3b44-7d1f-440d-bbce-84f32dd4d6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:39.133985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c3b44-7d1f-440d-bbce-84f32dd4d6d3 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:39.135377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 281/542] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:39.143081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181244c-9244-49fe-b491-3ce6347ddeec None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:39.143391 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181244c-9244-49fe-b491-3ce6347ddeec None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:39.143644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181244c-9244-49fe-b491-3ce6347ddeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:39.175035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-9181244c-9244-49fe-b491-3ce6347ddeec None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:39.177165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9181244c-9244-49fe-b491-3ce6347ddeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:39.319435 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 262/543] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:39.355105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c3b44-7d1f-440d-bbce-84f32dd4d6d3 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:39.492836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c3b44-7d1f-440d-bbce-84f32dd4d6d3 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:39.546745 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b526c47-e37f-4593-945c-a0ca1556804c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:39.547573 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b526c47-e37f-4593-945c-a0ca1556804c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:39.547859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b526c47-e37f-4593-945c-a0ca1556804c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:39.548142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b526c47-e37f-4593-945c-a0ca1556804c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:39.551199 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf45c4-b445-4b3d-a6fc-f52f00b42366 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:39.551491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf45c4-b445-4b3d-a6fc-f52f00b42366 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:39.551778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf45c4-b445-4b3d-a6fc-f52f00b42366 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:39.553197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 263/544] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:39.558959 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b23100-cfbc-4570-9120-c4e900afff89 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:39.561372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b23100-cfbc-4570-9120-c4e900afff89 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:39.561835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b23100-cfbc-4570-9120-c4e900afff89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:39.564489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b526c47-e37f-4593-945c-a0ca1556804c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:39.575147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-3b526c47-e37f-4593-945c-a0ca1556804c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:39.579957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:39.580941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:39.581740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:39.581740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:39.581740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:39.581740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:39.581740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:39.581740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:39.581740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:39.581740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:39.581740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:39.581740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:39.581740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:39.581740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:39.584914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 282/545] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:39.587638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b526c47-e37f-4593-945c-a0ca1556804c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:39.598073 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-82b23100-cfbc-4570-9120-c4e900afff89 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:39.599569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-82b23100-cfbc-4570-9120-c4e900afff89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:39.697752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 264/546] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 15 10:02:39.758384 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b23100-cfbc-4570-9120-c4e900afff89 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:39.784132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b526c47-e37f-4593-945c-a0ca1556804c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:39.853775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:39.866211 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e93cf-1646-4daf-8ec1-2e275daa7be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:39.867282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e93cf-1646-4daf-8ec1-2e275daa7be0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:39.867728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e93cf-1646-4daf-8ec1-2e275daa7be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:39.868319 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e93cf-1646-4daf-8ec1-2e275daa7be0 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:39.869258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e93cf-1646-4daf-8ec1-2e275daa7be0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:39.869755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e93cf-1646-4daf-8ec1-2e275daa7be0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:39.871385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e93cf-1646-4daf-8ec1-2e275daa7be0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:39.892298 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 283/547] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:39.899164 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e93cf-1646-4daf-8ec1-2e275daa7be0 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:39.973933 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b23100-cfbc-4570-9120-c4e900afff89 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:40.037951 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:40.063194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc51b34-6e8b-4937-95cf-3384430405e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:40.064116 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc51b34-6e8b-4937-95cf-3384430405e2 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:40.064467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc51b34-6e8b-4937-95cf-3384430405e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:40.064816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc51b34-6e8b-4937-95cf-3384430405e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:40.083982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e93cf-1646-4daf-8ec1-2e275daa7be0 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:40.094416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc51b34-6e8b-4937-95cf-3384430405e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-cbc51b34-6e8b-4937-95cf-3384430405e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:40.102313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:40.103288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:40.104050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:40.104050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:40.104050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:40.104050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:40.104050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:40.104050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:40.104050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:40.104050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:40.104050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:40.104050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:40.104050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:40.104050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:40.113412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 265/548] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:40.119205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc51b34-6e8b-4937-95cf-3384430405e2 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:40.133203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041b873-9f0e-4647-b48d-4a2e24cedc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:40.134316 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041b873-9f0e-4647-b48d-4a2e24cedc93 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:40.134889 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041b873-9f0e-4647-b48d-4a2e24cedc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:40.135364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041b873-9f0e-4647-b48d-4a2e24cedc93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:40.155663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041b873-9f0e-4647-b48d-4a2e24cedc93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:40.158936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041b873-9f0e-4647-b48d-4a2e24cedc93 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:40.195242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-5041b873-9f0e-4647-b48d-4a2e24cedc93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:40.195242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:40.195242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:40.195242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:40.195242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:40.195242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:40.195242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:40.195242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:40.195242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:40.195242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:40.195242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:40.195242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:40.195242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:40.195242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:40.195242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:40.196184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:40.197542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:40.197542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:40.197542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:40.197542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:40.197542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 284/549] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:40.203496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041b873-9f0e-4647-b48d-4a2e24cedc93 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:40.295097 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc51b34-6e8b-4937-95cf-3384430405e2 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:40.319077 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:40.338222 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041b873-9f0e-4647-b48d-4a2e24cedc93 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:40.355184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:40.379146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2409e-a5e7-4a3d-9ad9-9ab9dc99a7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:40.380189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2409e-a5e7-4a3d-9ad9-9ab9dc99a7fb None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:40.380695 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2409e-a5e7-4a3d-9ad9-9ab9dc99a7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:40.381197 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2409e-a5e7-4a3d-9ad9-9ab9dc99a7fb None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:40.382124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2409e-a5e7-4a3d-9ad9-9ab9dc99a7fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:40.382629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2409e-a5e7-4a3d-9ad9-9ab9dc99a7fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:40.383216 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1bdad0-ab26-4730-b08d-2001ae77cff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:40.383889 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bdad0-ab26-4730-b08d-2001ae77cff1 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:40.384213 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bdad0-ab26-4730-b08d-2001ae77cff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:40.384491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bdad0-ab26-4730-b08d-2001ae77cff1 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:40.385183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2409e-a5e7-4a3d-9ad9-9ab9dc99a7fb None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:40.385302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1bdad0-ab26-4730-b08d-2001ae77cff1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:40.385562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1bdad0-ab26-4730-b08d-2001ae77cff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:40.386932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1bdad0-ab26-4730-b08d-2001ae77cff1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:40.398256 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 266/550] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:40.400617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 285/551] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:40.407442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1bdad0-ab26-4730-b08d-2001ae77cff1 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:40.413074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2409e-a5e7-4a3d-9ad9-9ab9dc99a7fb None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:40.572390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1bdad0-ab26-4730-b08d-2001ae77cff1 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:40.607243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2409e-a5e7-4a3d-9ad9-9ab9dc99a7fb None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:40.623927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa73e3-fa11-49bf-9dc9-6a04621037fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:40.625418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa73e3-fa11-49bf-9dc9-6a04621037fe None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:40.626741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa73e3-fa11-49bf-9dc9-6a04621037fe None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:40.627321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa73e3-fa11-49bf-9dc9-6a04621037fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:40.652881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa73e3-fa11-49bf-9dc9-6a04621037fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:40.654709 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa73e3-fa11-49bf-9dc9-6a04621037fe None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:40.677524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a76f81-baf6-448c-aa55-6b476ac55fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:40.678320 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a76f81-baf6-448c-aa55-6b476ac55fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:40.678607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a76f81-baf6-448c-aa55-6b476ac55fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:40.678884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a76f81-baf6-448c-aa55-6b476ac55fd7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:40.683358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-b5fa73e3-fa11-49bf-9dc9-6a04621037fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:40.683358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:40.683358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:40.683358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:40.683358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:40.683358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:40.683358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:40.683358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:40.683358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:40.683358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:40.683358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:40.683358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:40.683358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:40.683358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:40.683358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:40.684196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:40.685084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:40.685084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:40.685084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:40.685084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:40.687298 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a76f81-baf6-448c-aa55-6b476ac55fd7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:40.688370 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 267/552] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:40.688830 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a76f81-baf6-448c-aa55-6b476ac55fd7 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:40.694511 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8edc7-9d97-482b-a2b9-c887de068acb None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:40.694647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8edc7-9d97-482b-a2b9-c887de068acb None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:40.694854 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8edc7-9d97-482b-a2b9-c887de068acb None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:40.696321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 268/553] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:40.697745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8edc7-9d97-482b-a2b9-c887de068acb None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-41a76f81-baf6-448c-aa55-6b476ac55fd7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:40.716431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:40.718003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:40.718910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:40.718910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:40.718910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:40.718910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:40.718910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:40.718910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:40.718910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:40.718910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:40.718910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:02:40.718910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:40.721048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 286/554] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:02:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:40.723808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdd6ef-c867-4b9d-8393-25d9b6ac3be8 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:40.725375 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdd6ef-c867-4b9d-8393-25d9b6ac3be8 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:40.725775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdd6ef-c867-4b9d-8393-25d9b6ac3be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:40.736515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:40.748369 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-acbdd6ef-c867-4b9d-8393-25d9b6ac3be8 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:40.749822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-acbdd6ef-c867-4b9d-8393-25d9b6ac3be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:40.789444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:40.848236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8edc7-9d97-482b-a2b9-c887de068acb None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:40.850655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 287/555] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:40.852574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdd6ef-c867-4b9d-8393-25d9b6ac3be8 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:40.907485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd965c78-52bd-4212-9ed5-704c0888b3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:40.909302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd965c78-52bd-4212-9ed5-704c0888b3ce None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:40.910137 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd965c78-52bd-4212-9ed5-704c0888b3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:40.910722 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd965c78-52bd-4212-9ed5-704c0888b3ce None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:40.912539 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd965c78-52bd-4212-9ed5-704c0888b3ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:40.914018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd965c78-52bd-4212-9ed5-704c0888b3ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:40.915681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd965c78-52bd-4212-9ed5-704c0888b3ce None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:40.941678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 269/556] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:40.941678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd965c78-52bd-4212-9ed5-704c0888b3ce None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:41.002246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdd6ef-c867-4b9d-8393-25d9b6ac3be8 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:41.048039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18949336-0f7c-4290-8048-ff1caf19bd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:41.049280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18949336-0f7c-4290-8048-ff1caf19bd56 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:41.049773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18949336-0f7c-4290-8048-ff1caf19bd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:41.050322 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18949336-0f7c-4290-8048-ff1caf19bd56 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:41.051798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18949336-0f7c-4290-8048-ff1caf19bd56 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:41.052613 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:41.053181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18949336-0f7c-4290-8048-ff1caf19bd56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:41.055075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18949336-0f7c-4290-8048-ff1caf19bd56 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:41.073553 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd965c78-52bd-4212-9ed5-704c0888b3ce None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:41.076188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 288/557] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Jun 15 10:02:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:41.078245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18949336-0f7c-4290-8048-ff1caf19bd56 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:41.107947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:41.118483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b24031-e40e-4dd8-9bca-cf0debc83c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:41.126266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b24031-e40e-4dd8-9bca-cf0debc83c94 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:41.126266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b24031-e40e-4dd8-9bca-cf0debc83c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:41.126266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b24031-e40e-4dd8-9bca-cf0debc83c94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:41.132617 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b24031-e40e-4dd8-9bca-cf0debc83c94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-c6b24031-e40e-4dd8-9bca-cf0debc83c94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:41.141250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:41.142697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:41.143546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:41.143546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:41.143546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:41.143546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:41.143546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:41.143546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:41.143546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:41.143546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:41.143546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:41.143546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:41.143546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:41.143546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:41.143546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 270/558] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:41.143546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b24031-e40e-4dd8-9bca-cf0debc83c94 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:41.205940 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18949336-0f7c-4290-8048-ff1caf19bd56 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:41.248404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2123a-d2ba-4577-b922-b3340d01bd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:41.250319 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2123a-d2ba-4577-b922-b3340d01bd25 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:41.250922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2123a-d2ba-4577-b922-b3340d01bd25 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:41.251494 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce2123a-d2ba-4577-b922-b3340d01bd25 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:41.261859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2123a-d2ba-4577-b922-b3340d01bd25 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:41.263327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce2123a-d2ba-4577-b922-b3340d01bd25 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:41.267331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b24031-e40e-4dd8-9bca-cf0debc83c94 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-8ce2123a-d2ba-4577-b922-b3340d01bd25 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:41.287584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:41.288552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:41.289607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:41.289607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:41.289607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:41.289607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:41.289607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:41.289607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:41.289607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:41.289607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:41.289607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:02:41.289607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:41.296026 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 289/559] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jun 15 10:02:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:41.296026 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2123a-d2ba-4577-b922-b3340d01bd25 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:41.312209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c30cc-e2af-4b8d-a9ad-d82bb670d736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:41.313026 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c30cc-e2af-4b8d-a9ad-d82bb670d736 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:41.313293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c30cc-e2af-4b8d-a9ad-d82bb670d736 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:41.313572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c30cc-e2af-4b8d-a9ad-d82bb670d736 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:41.328886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:41.397506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c30cc-e2af-4b8d-a9ad-d82bb670d736 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:41.398881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c30cc-e2af-4b8d-a9ad-d82bb670d736 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:41.420102 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:41.441514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce2123a-d2ba-4577-b922-b3340d01bd25 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:41.482679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d635e9a-bdcd-47c0-959a-1bee66894b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:41.483962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d635e9a-bdcd-47c0-959a-1bee66894b81 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:41.484245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d635e9a-bdcd-47c0-959a-1bee66894b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:41.484524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d635e9a-bdcd-47c0-959a-1bee66894b81 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:41.485352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d635e9a-bdcd-47c0-959a-1bee66894b81 None admin] RBAC: Adding query filter params (name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:41.485666 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d635e9a-bdcd-47c0-959a-1bee66894b81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:41.487261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d635e9a-bdcd-47c0-959a-1bee66894b81 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:41.498195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 290/560] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:41.499082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d635e9a-bdcd-47c0-959a-1bee66894b81 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:41.524310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 271/561] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:41.527642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5558d-a769-4200-890d-760851d66eb9 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:41.527642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5558d-a769-4200-890d-760851d66eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:41.527642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5558d-a769-4200-890d-760851d66eb9 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:41.532465 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 272/562] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:41.533458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5558d-a769-4200-890d-760851d66eb9 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:41.578289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:41.613495 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d635e9a-bdcd-47c0-959a-1bee66894b81 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:41.643252 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5558d-a769-4200-890d-760851d66eb9 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:41.646794 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87309bdb-f390-491a-b2bc-96116951fe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:41.647465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87309bdb-f390-491a-b2bc-96116951fe1c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:41.647796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87309bdb-f390-491a-b2bc-96116951fe1c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:41.648029 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87309bdb-f390-491a-b2bc-96116951fe1c None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:41.648866 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87309bdb-f390-491a-b2bc-96116951fe1c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:41.649204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87309bdb-f390-491a-b2bc-96116951fe1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:41.650631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87309bdb-f390-491a-b2bc-96116951fe1c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:41.669325 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 291/563] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jun 15 10:02:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:41.670158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87309bdb-f390-491a-b2bc-96116951fe1c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:41.673442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:41.680510 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505a8a8a-78e7-4e35-a23d-5510cfdfe5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:41.681353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a8a8a-78e7-4e35-a23d-5510cfdfe5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:41.682904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a8a8a-78e7-4e35-a23d-5510cfdfe5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:41.682904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a8a8a-78e7-4e35-a23d-5510cfdfe5d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:41.692701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a8a8a-78e7-4e35-a23d-5510cfdfe5d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:41.694175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505a8a8a-78e7-4e35-a23d-5510cfdfe5d6 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:41.711879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-505a8a8a-78e7-4e35-a23d-5510cfdfe5d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:41.711879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:41.711879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:41.711879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:41.711879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:41.711879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:41.711879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:41.711879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:41.711879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:41.711879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:41.711879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:41.711879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:41.711879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:41.711879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:41.711879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:41.712931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:41.713998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:41.713998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:41.713998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:41.713998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:41.717714 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 273/564] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:41.718917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324d7ca-1c86-4648-bde5-c332dbee084f None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:41.719207 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324d7ca-1c86-4648-bde5-c332dbee084f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:41.719485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324d7ca-1c86-4648-bde5-c332dbee084f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:41.735670 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-f324d7ca-1c86-4648-bde5-c332dbee084f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:41.736873 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f324d7ca-1c86-4648-bde5-c332dbee084f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:41.770639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87309bdb-f390-491a-b2bc-96116951fe1c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:41.802182 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 274/565] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:41.803503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc38d6-f406-4b83-a4b7-4539438c56d6 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:41.803787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc38d6-f406-4b83-a4b7-4539438c56d6 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:41.804063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc38d6-f406-4b83-a4b7-4539438c56d6 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:41.805362 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 275/566] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:41.806049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc38d6-f406-4b83-a4b7-4539438c56d6 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:41.814247 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac6827-4c37-4ec0-bee5-afdbe1a84b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:41.815060 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac6827-4c37-4ec0-bee5-afdbe1a84b8b None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:41.815343 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac6827-4c37-4ec0-bee5-afdbe1a84b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:41.815618 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac6827-4c37-4ec0-bee5-afdbe1a84b8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:41.827926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac6827-4c37-4ec0-bee5-afdbe1a84b8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:41.829416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac6827-4c37-4ec0-bee5-afdbe1a84b8b None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:41.832759 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:41.847349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-d1ac6827-4c37-4ec0-bee5-afdbe1a84b8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:41.847349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:41.847349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:41.847349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:41.847349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:41.847349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:41.847349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:41.847349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:41.847349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:41.847349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:41.847349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:41.847349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:41.847349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:41.847349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:41.847349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:41.848439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:41.854350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:41.854350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:41.854350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:41.854350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:41.854350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 292/567] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:41.854350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac6827-4c37-4ec0-bee5-afdbe1a84b8b None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:41.910313 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:41.912204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc38d6-f406-4b83-a4b7-4539438c56d6 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:41.948699 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f97a4-aa52-48c2-b531-9280ed060854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:41.949594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f97a4-aa52-48c2-b531-9280ed060854 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:41.949948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f97a4-aa52-48c2-b531-9280ed060854 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:41.950226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f97a4-aa52-48c2-b531-9280ed060854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:41.956387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac6827-4c37-4ec0-bee5-afdbe1a84b8b None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:41.960251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f97a4-aa52-48c2-b531-9280ed060854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:41.961839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f97a4-aa52-48c2-b531-9280ed060854 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:41.980081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-d49f97a4-aa52-48c2-b531-9280ed060854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:41.980081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:41.980081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:41.980081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:41.980081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:41.980081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:41.980081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:41.980081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:41.980081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:41.980081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:41.980081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:41.980081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:41.980081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:41.980081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:41.980081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:41.981001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:41.981879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:41.981879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:41.981879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:41.981879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:41.981879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 276/568] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:41.982514 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940483ec-18b4-4bfa-b5b9-f72cf0cfac0b None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:41.982803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940483ec-18b4-4bfa-b5b9-f72cf0cfac0b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:41.983728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940483ec-18b4-4bfa-b5b9-f72cf0cfac0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:41.996328 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3a270-de71-48ea-b26a-5a6307651336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:41.997017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3a270-de71-48ea-b26a-5a6307651336 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:41.997304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3a270-de71-48ea-b26a-5a6307651336 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:41.997580 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3a270-de71-48ea-b26a-5a6307651336 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:41.998312 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3a270-de71-48ea-b26a-5a6307651336 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:41.999312 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3a270-de71-48ea-b26a-5a6307651336 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:42.000794 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3a270-de71-48ea-b26a-5a6307651336 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:42.001808 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-940483ec-18b4-4bfa-b5b9-f72cf0cfac0b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:42.003033 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-940483ec-18b4-4bfa-b5b9-f72cf0cfac0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:42.013044 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 293/569] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:42.013769 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3a270-de71-48ea-b26a-5a6307651336 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:42.075861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 277/570] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:42.078197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940483ec-18b4-4bfa-b5b9-f72cf0cfac0b None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:42.086660 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:42.118639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3a270-de71-48ea-b26a-5a6307651336 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:42.132870 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:42.152718 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d1f08-cefb-483f-8cdb-e02ae9d20b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:42.153554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d1f08-cefb-483f-8cdb-e02ae9d20b18 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:42.153833 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d1f08-cefb-483f-8cdb-e02ae9d20b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:42.154108 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d1f08-cefb-483f-8cdb-e02ae9d20b18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:42.165673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d1f08-cefb-483f-8cdb-e02ae9d20b18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-688d1f08-cefb-483f-8cdb-e02ae9d20b18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:42.174338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:42.175437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:42.176533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:42.176533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:42.176533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:42.176533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:42.176533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:42.176533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:42.176533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:42.176533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:42.176533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:42.176533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:42.176533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:42.176533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:42.176533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 294/571] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:42.177242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d1f08-cefb-483f-8cdb-e02ae9d20b18 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:42.185538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940483ec-18b4-4bfa-b5b9-f72cf0cfac0b None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:42.220846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d74a0-00fa-4d87-a83c-d79877fdeb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:42.221531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d74a0-00fa-4d87-a83c-d79877fdeb3d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:42.228029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d74a0-00fa-4d87-a83c-d79877fdeb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:42.228029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d74a0-00fa-4d87-a83c-d79877fdeb3d None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:42.228029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d74a0-00fa-4d87-a83c-d79877fdeb3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:42.228029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d74a0-00fa-4d87-a83c-d79877fdeb3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:42.228029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d74a0-00fa-4d87-a83c-d79877fdeb3d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:42.236499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 278/572] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:42.237324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d74a0-00fa-4d87-a83c-d79877fdeb3d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:42.278732 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d1f08-cefb-483f-8cdb-e02ae9d20b18 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:42.312371 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:42.319436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afd643-2dab-4e62-8ddc-cbeda1cefabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:42.320116 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afd643-2dab-4e62-8ddc-cbeda1cefabe None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:42.320402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afd643-2dab-4e62-8ddc-cbeda1cefabe None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:42.320685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20afd643-2dab-4e62-8ddc-cbeda1cefabe None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:42.325280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afd643-2dab-4e62-8ddc-cbeda1cefabe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:42.325280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afd643-2dab-4e62-8ddc-cbeda1cefabe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:42.325280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20afd643-2dab-4e62-8ddc-cbeda1cefabe None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:42.336126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 295/573] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:42.336920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afd643-2dab-4e62-8ddc-cbeda1cefabe None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:42.350864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d74a0-00fa-4d87-a83c-d79877fdeb3d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:42.374817 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:42.386722 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d95c74-4e3b-4013-b7f8-0c2aaf2d7905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:42.387521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d95c74-4e3b-4013-b7f8-0c2aaf2d7905 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:42.387804 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d95c74-4e3b-4013-b7f8-0c2aaf2d7905 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:42.388084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d95c74-4e3b-4013-b7f8-0c2aaf2d7905 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:42.396524 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d95c74-4e3b-4013-b7f8-0c2aaf2d7905 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:42.398136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d95c74-4e3b-4013-b7f8-0c2aaf2d7905 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-19d95c74-4e3b-4013-b7f8-0c2aaf2d7905 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:42.413335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:42.414366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:42.415277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:42.415277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:42.415277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:42.415277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:42.415277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:42.415277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:42.415277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:42.415277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:42.415277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 10:02:42.415277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:42.415277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 279/574] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:42.415277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d95c74-4e3b-4013-b7f8-0c2aaf2d7905 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:42.444855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20afd643-2dab-4e62-8ddc-cbeda1cefabe None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:42.479747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32a2f9-de0d-4ec2-80d0-e59d98dd581a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:42.480536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32a2f9-de0d-4ec2-80d0-e59d98dd581a None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:42.480842 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32a2f9-de0d-4ec2-80d0-e59d98dd581a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:42.481124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32a2f9-de0d-4ec2-80d0-e59d98dd581a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:42.491819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32a2f9-de0d-4ec2-80d0-e59d98dd581a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-dd32a2f9-de0d-4ec2-80d0-e59d98dd581a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:42.499993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:42.500907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:42.501914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:42.501914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:42.501914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:42.501914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:42.501914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:42.501914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:42.501914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:42.501914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:42.501914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:42.501914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:42.501914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:42.501914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:42.501914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 296/575] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:42.502595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32a2f9-de0d-4ec2-80d0-e59d98dd581a None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:42.522641 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d95c74-4e3b-4013-b7f8-0c2aaf2d7905 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:42.549983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:42.559264 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fb74d-1281-44f2-80c8-8b755e3498ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:42.565663 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fb74d-1281-44f2-80c8-8b755e3498ea None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:42.565663 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fb74d-1281-44f2-80c8-8b755e3498ea None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:42.565663 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fb74d-1281-44f2-80c8-8b755e3498ea None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:42.565663 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fb74d-1281-44f2-80c8-8b755e3498ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:42.565663 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fb74d-1281-44f2-80c8-8b755e3498ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:42.565663 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fb74d-1281-44f2-80c8-8b755e3498ea None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:42.571683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 280/576] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:42.572600 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fb74d-1281-44f2-80c8-8b755e3498ea None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:42.604070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:42.610691 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32a2f9-de0d-4ec2-80d0-e59d98dd581a None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:42.645023 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee24191d-bf5f-40bd-a50b-e24b58256200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:42.645822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24191d-bf5f-40bd-a50b-e24b58256200 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:42.646106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24191d-bf5f-40bd-a50b-e24b58256200 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:42.646383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24191d-bf5f-40bd-a50b-e24b58256200 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:42.655874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee24191d-bf5f-40bd-a50b-e24b58256200 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:42.655874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee24191d-bf5f-40bd-a50b-e24b58256200 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:42.671251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-ee24191d-bf5f-40bd-a50b-e24b58256200 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:42.671251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:42.671251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:42.671251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:42.671251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:42.671251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:42.671251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:42.671251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:42.671251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:42.671251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:42.671251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:42.671251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:42.671251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:42.671251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:42.671251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:42.672367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:42.673311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:42.673311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:42.673311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:42.673311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:42.673311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:42.673311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:42.673311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:42.673311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:42.673311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 297/577] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:42.673311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee24191d-bf5f-40bd-a50b-e24b58256200 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:42.677653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fb74d-1281-44f2-80c8-8b755e3498ea None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:42.713429 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e89141-04e7-4b5c-9c84-12fa005f098f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:42.714223 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e89141-04e7-4b5c-9c84-12fa005f098f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:42.714500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e89141-04e7-4b5c-9c84-12fa005f098f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:42.714767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e89141-04e7-4b5c-9c84-12fa005f098f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:42.722432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e89141-04e7-4b5c-9c84-12fa005f098f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:42.723877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e89141-04e7-4b5c-9c84-12fa005f098f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:42.738829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-64e89141-04e7-4b5c-9c84-12fa005f098f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:42.738829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:42.738829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:42.738829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:42.738829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:42.738829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:42.738829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:42.738829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:42.738829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:42.738829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:42.738829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:42.738829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:42.738829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:42.738829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:42.738829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:42.739786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:42.740839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:42.740839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:42.740839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:42.740839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:42.740839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:42.740839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:42.740839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:42.740839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:42.740839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 281/578] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:42.741353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e89141-04e7-4b5c-9c84-12fa005f098f None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:42.782199 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee24191d-bf5f-40bd-a50b-e24b58256200 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:42.792031 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:42.816897 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d9a0a-0bee-4ae9-ac8b-6b7f4149b4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:42.817608 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d9a0a-0bee-4ae9-ac8b-6b7f4149b4ef None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:42.817889 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d9a0a-0bee-4ae9-ac8b-6b7f4149b4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:42.818167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d9a0a-0bee-4ae9-ac8b-6b7f4149b4ef None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:42.819089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d9a0a-0bee-4ae9-ac8b-6b7f4149b4ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:42.819404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d9a0a-0bee-4ae9-ac8b-6b7f4149b4ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:42.820842 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d9a0a-0bee-4ae9-ac8b-6b7f4149b4ef None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:42.839076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 298/579] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Jun 15 10:02:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:42.839899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d9a0a-0bee-4ae9-ac8b-6b7f4149b4ef None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:42.843116 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e89141-04e7-4b5c-9c84-12fa005f098f None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:42.854377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:42.876807 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628532e-5bb0-4966-a11f-d5d2131c88a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:42.877501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628532e-5bb0-4966-a11f-d5d2131c88a8 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:42.877780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628532e-5bb0-4966-a11f-d5d2131c88a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:42.878111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628532e-5bb0-4966-a11f-d5d2131c88a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:42.878798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628532e-5bb0-4966-a11f-d5d2131c88a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:42.879102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628532e-5bb0-4966-a11f-d5d2131c88a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:42.880547 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628532e-5bb0-4966-a11f-d5d2131c88a8 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:42.888677 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 282/580] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:42.889696 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628532e-5bb0-4966-a11f-d5d2131c88a8 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:42.941554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d9a0a-0bee-4ae9-ac8b-6b7f4149b4ef None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:42.978341 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3ac53-3836-4363-bf78-29af3e2c1375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:42.979078 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3ac53-3836-4363-bf78-29af3e2c1375 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:42.979365 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3ac53-3836-4363-bf78-29af3e2c1375 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:42.979646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3ac53-3836-4363-bf78-29af3e2c1375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:42.989037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628532e-5bb0-4966-a11f-d5d2131c88a8 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:42.990631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3ac53-3836-4363-bf78-29af3e2c1375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:42.992084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a3ac53-3836-4363-bf78-29af3e2c1375 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:42.999413 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:43.009997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-32a3ac53-3836-4363-bf78-29af3e2c1375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:43.009997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:43.009997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:43.009997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:43.009997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:43.009997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:43.009997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:43.009997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:43.009997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:43.009997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:43.009997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:43.009997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:43.009997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:43.009997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:43.009997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:43.011049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:43.011933 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:43.011933 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:43.011933 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:43.011933 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:43.011933 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 299/581] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:43.012580 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3ac53-3836-4363-bf78-29af3e2c1375 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:43.027482 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2582ca-a019-4f23-a648-850f8423a738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:43.028252 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2582ca-a019-4f23-a648-850f8423a738 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:43.028540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2582ca-a019-4f23-a648-850f8423a738 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:43.028815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2582ca-a019-4f23-a648-850f8423a738 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:43.030163 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2582ca-a019-4f23-a648-850f8423a738 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:43.030534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2582ca-a019-4f23-a648-850f8423a738 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:43.031990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2582ca-a019-4f23-a648-850f8423a738 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:43.043783 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 283/582] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:43.044605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2582ca-a019-4f23-a648-850f8423a738 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:43.076300 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:43.113912 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a3ac53-3836-4363-bf78-29af3e2c1375 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:43.146501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2582ca-a019-4f23-a648-850f8423a738 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:43.148221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99896db3-c48e-4acc-9102-71a441dfdf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:43.148883 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99896db3-c48e-4acc-9102-71a441dfdf08 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:43.149197 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99896db3-c48e-4acc-9102-71a441dfdf08 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:43.149477 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99896db3-c48e-4acc-9102-71a441dfdf08 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:43.150273 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99896db3-c48e-4acc-9102-71a441dfdf08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:43.150586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99896db3-c48e-4acc-9102-71a441dfdf08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:43.152022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99896db3-c48e-4acc-9102-71a441dfdf08 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:43.168999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 300/583] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:43.168999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99896db3-c48e-4acc-9102-71a441dfdf08 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:43.187582 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0af77-dfaf-4eec-9b77-129fd4f18887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:43.188444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0af77-dfaf-4eec-9b77-129fd4f18887 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:43.188733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0af77-dfaf-4eec-9b77-129fd4f18887 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:43.189037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0af77-dfaf-4eec-9b77-129fd4f18887 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:43.199794 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0af77-dfaf-4eec-9b77-129fd4f18887 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:43.201381 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d0af77-dfaf-4eec-9b77-129fd4f18887 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:43.221004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-72d0af77-dfaf-4eec-9b77-129fd4f18887 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:43.221004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:43.221004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:43.221004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:43.221004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:43.221004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:43.221004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:43.221004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:43.221004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:43.221004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:43.221004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:43.221004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:43.221004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:43.221004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:43.221004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:43.222099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:43.223191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:43.223191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:43.223191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:43.223191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:43.223581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 284/584] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:43.224570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0af77-dfaf-4eec-9b77-129fd4f18887 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:43.249924 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:43.269356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99896db3-c48e-4acc-9102-71a441dfdf08 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:43.293496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:43.304354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49be04c-e8f8-486d-ad5b-07ad9a9e4797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:43.305190 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49be04c-e8f8-486d-ad5b-07ad9a9e4797 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:43.305472 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49be04c-e8f8-486d-ad5b-07ad9a9e4797 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:43.305747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49be04c-e8f8-486d-ad5b-07ad9a9e4797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:43.316680 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49be04c-e8f8-486d-ad5b-07ad9a9e4797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:43.318162 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49be04c-e8f8-486d-ad5b-07ad9a9e4797 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:43.328200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0af77-dfaf-4eec-9b77-129fd4f18887 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:43.335632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-c49be04c-e8f8-486d-ad5b-07ad9a9e4797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:43.335632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:43.335632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:43.335632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:43.335632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:43.335632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:43.335632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:43.335632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:43.335632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:43.335632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:43.335632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:43.335632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:43.335632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:43.335632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:43.335632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:43.336642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:43.343101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:43.343101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:43.343101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:43.343101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:43.343101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 301/585] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:43.343101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49be04c-e8f8-486d-ad5b-07ad9a9e4797 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:43.363141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83592da7-18ab-471e-bee0-2b9d6339049d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:43.363808 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83592da7-18ab-471e-bee0-2b9d6339049d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:43.364090 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83592da7-18ab-471e-bee0-2b9d6339049d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:43.364367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83592da7-18ab-471e-bee0-2b9d6339049d None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:43.365127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83592da7-18ab-471e-bee0-2b9d6339049d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:43.365441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83592da7-18ab-471e-bee0-2b9d6339049d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:43.366859 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83592da7-18ab-471e-bee0-2b9d6339049d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:43.378071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 285/586] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:43.378863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83592da7-18ab-471e-bee0-2b9d6339049d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:43.440700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49be04c-e8f8-486d-ad5b-07ad9a9e4797 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:43.465469 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:43.474738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311ff53-2434-48f7-bc3a-d71cc29251e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:43.478906 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311ff53-2434-48f7-bc3a-d71cc29251e1 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:43.478906 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311ff53-2434-48f7-bc3a-d71cc29251e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:43.478906 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311ff53-2434-48f7-bc3a-d71cc29251e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:43.478906 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311ff53-2434-48f7-bc3a-d71cc29251e1 None admin] RBAC: Adding query filter params (role.id=2c276d442f7f4dd59ae7fc3a03943716, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=589e66c6228a44adbc9e418794d7d91b) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:43.478906 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311ff53-2434-48f7-bc3a-d71cc29251e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:43.478906 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311ff53-2434-48f7-bc3a-d71cc29251e1 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:43.483592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83592da7-18ab-471e-bee0-2b9d6339049d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:43.484317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 302/587] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:02:43 2022] GET /identity/v3/role_assignments?user.id=589e66c6228a44adbc9e418794d7d91b&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=2c276d442f7f4dd59ae7fc3a03943716 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:43.484929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311ff53-2434-48f7-bc3a-d71cc29251e1 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:43.526953 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121b9b9-c11e-4954-a027-b58029541091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:43.528110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121b9b9-c11e-4954-a027-b58029541091 None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:43.528458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121b9b9-c11e-4954-a027-b58029541091 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:43.528777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121b9b9-c11e-4954-a027-b58029541091 None admin] PATH_INFO: `/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/26f23f8595fc43ad94aee7d0192164a1/roles/2c276d442f7f4dd59ae7fc3a03943716` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:43.539743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:43.550167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121b9b9-c11e-4954-a027-b58029541091 None admin] RBAC: Authorizing `identity:create_grant(project_id=db47f1bb607c4d1a9c346fdc80ecadfa, user_id=26f23f8595fc43ad94aee7d0192164a1, role_id=2c276d442f7f4dd59ae7fc3a03943716)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:43.551932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121b9b9-c11e-4954-a027-b58029541091 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:43.605136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311ff53-2434-48f7-bc3a-d71cc29251e1 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:43.640996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 286/588] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:02:43 2022] PUT /identity/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/26f23f8595fc43ad94aee7d0192164a1/roles/2c276d442f7f4dd59ae7fc3a03943716 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:02:43.640996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121b9b9-c11e-4954-a027-b58029541091 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:43.658100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a2a4f-b8b9-4b6f-a55a-5340336ce938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:43.660226 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a2a4f-b8b9-4b6f-a55a-5340336ce938 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:43.660525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a2a4f-b8b9-4b6f-a55a-5340336ce938 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:43.661524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a2a4f-b8b9-4b6f-a55a-5340336ce938 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:43.661783 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a2a4f-b8b9-4b6f-a55a-5340336ce938 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:43.662092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a2a4f-b8b9-4b6f-a55a-5340336ce938 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:43.663542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a2a4f-b8b9-4b6f-a55a-5340336ce938 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:43.684605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 303/589] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:43.684605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a2a4f-b8b9-4b6f-a55a-5340336ce938 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:43.801877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121b9b9-c11e-4954-a027-b58029541091 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:43.822747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:43.827568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a2a4f-b8b9-4b6f-a55a-5340336ce938 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:43.844894 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:43.852442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182406bc-5b19-41af-bc5f-e6168db89e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:43.853401 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182406bc-5b19-41af-bc5f-e6168db89e74 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:43.853784 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182406bc-5b19-41af-bc5f-e6168db89e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:43.854160 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182406bc-5b19-41af-bc5f-e6168db89e74 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:43.855002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182406bc-5b19-41af-bc5f-e6168db89e74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:43.855437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182406bc-5b19-41af-bc5f-e6168db89e74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:43.856905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182406bc-5b19-41af-bc5f-e6168db89e74 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:43.872756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 287/590] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:43.873744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182406bc-5b19-41af-bc5f-e6168db89e74 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:43.876058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821a8e8-c44b-4385-99f4-07e49f9e47d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:43.877093 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821a8e8-c44b-4385-99f4-07e49f9e47d7 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:43.877376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821a8e8-c44b-4385-99f4-07e49f9e47d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:43.878513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821a8e8-c44b-4385-99f4-07e49f9e47d7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:43.887839 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821a8e8-c44b-4385-99f4-07e49f9e47d7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:43.890273 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821a8e8-c44b-4385-99f4-07e49f9e47d7 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-a821a8e8-c44b-4385-99f4-07e49f9e47d7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:43.909469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:43.910639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:43.911779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:43.911779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:43.911779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:43.911779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:43.911779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:43.911779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:43.911779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:43.911779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:43.911779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 10:02:43.911779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:43.914818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 304/591] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:43.914818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821a8e8-c44b-4385-99f4-07e49f9e47d7 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:43.996192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182406bc-5b19-41af-bc5f-e6168db89e74 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:44.033990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7561c9-caaf-4628-b1c3-4f6db43d0dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:44.034853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7561c9-caaf-4628-b1c3-4f6db43d0dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:44.035134 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7561c9-caaf-4628-b1c3-4f6db43d0dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:44.035407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7561c9-caaf-4628-b1c3-4f6db43d0dc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:44.042785 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7561c9-caaf-4628-b1c3-4f6db43d0dc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:44.044226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7561c9-caaf-4628-b1c3-4f6db43d0dc1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:44.051605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821a8e8-c44b-4385-99f4-07e49f9e47d7 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:44.060748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-9e7561c9-caaf-4628-b1c3-4f6db43d0dc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:44.060748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:44.060748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:44.060748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:44.060748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:44.060748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:44.060748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:44.060748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:44.060748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:44.060748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:44.060748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:44.060748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:44.060748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:44.060748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:44.060748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:44.062032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:44.063136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:44.063136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:44.063136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:44.063136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:44.063136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:44.063136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:44.063136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:44.063136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:44.063136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 288/592] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:44.063835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7561c9-caaf-4628-b1c3-4f6db43d0dc1 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:44.101252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a499c0-7744-40e4-b95f-5102857a5199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:44.102052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a499c0-7744-40e4-b95f-5102857a5199 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:44.102330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a499c0-7744-40e4-b95f-5102857a5199 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:44.102598 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a499c0-7744-40e4-b95f-5102857a5199 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:44.112622 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a499c0-7744-40e4-b95f-5102857a5199 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:44.114100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a499c0-7744-40e4-b95f-5102857a5199 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:44.130678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-90a499c0-7744-40e4-b95f-5102857a5199 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:44.134950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:44.135905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:44.136815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:44.136815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:44.136815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:44.136815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:44.136815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:44.136815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:44.136815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:44.136815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:44.136815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:02:44.136815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:44.138802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 305/593] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:44.140485 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a499c0-7744-40e4-b95f-5102857a5199 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:44.165051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:44.193353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7561c9-caaf-4628-b1c3-4f6db43d0dc1 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:44.235499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5556569-5454-4081-abef-6a574e2712d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:44.236188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5556569-5454-4081-abef-6a574e2712d1 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:44.236476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5556569-5454-4081-abef-6a574e2712d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:44.236756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5556569-5454-4081-abef-6a574e2712d1 None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:44.237671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5556569-5454-4081-abef-6a574e2712d1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:44.237988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5556569-5454-4081-abef-6a574e2712d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:44.239462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5556569-5454-4081-abef-6a574e2712d1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:44.260342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 289/594] 10.222.0.60 () {56 vars in 1139 bytes} [Wed Jun 15 10:02:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:44.261369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5556569-5454-4081-abef-6a574e2712d1 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:44.271873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a499c0-7744-40e4-b95f-5102857a5199 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:44.310488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ea734-e135-4a13-bdb5-f46fd714fca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:44.312487 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ea734-e135-4a13-bdb5-f46fd714fca9 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:44.313120 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ea734-e135-4a13-bdb5-f46fd714fca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:44.313600 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ea734-e135-4a13-bdb5-f46fd714fca9 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:44.314825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ea734-e135-4a13-bdb5-f46fd714fca9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:44.315343 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ea734-e135-4a13-bdb5-f46fd714fca9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:44.317132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ea734-e135-4a13-bdb5-f46fd714fca9 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:44.331924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 306/595] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:44.333271 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ea734-e135-4a13-bdb5-f46fd714fca9 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:44.378477 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5556569-5454-4081-abef-6a574e2712d1 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:44.390739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:44.416863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00126b-1794-48b3-8e6f-15050c9400ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:44.421228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00126b-1794-48b3-8e6f-15050c9400ee None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:44.421228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00126b-1794-48b3-8e6f-15050c9400ee None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:44.421228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00126b-1794-48b3-8e6f-15050c9400ee None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:44.421228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00126b-1794-48b3-8e6f-15050c9400ee None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:44.421228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00126b-1794-48b3-8e6f-15050c9400ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:44.421589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00126b-1794-48b3-8e6f-15050c9400ee None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:44.431669 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:44.447565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 290/596] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Jun 15 10:02:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:44.448452 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00126b-1794-48b3-8e6f-15050c9400ee None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:44.454792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ea734-e135-4a13-bdb5-f46fd714fca9 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:44.494573 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00099b-6379-4d8a-acc4-84d2a154bc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:44.495824 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00099b-6379-4d8a-acc4-84d2a154bc71 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:44.496359 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00099b-6379-4d8a-acc4-84d2a154bc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:44.496896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00099b-6379-4d8a-acc4-84d2a154bc71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:44.509178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00099b-6379-4d8a-acc4-84d2a154bc71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:44.511349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00099b-6379-4d8a-acc4-84d2a154bc71 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:44.538742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-5f00099b-6379-4d8a-acc4-84d2a154bc71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:44.538742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:44.538742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:44.538742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:44.538742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:44.538742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:44.538742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:44.538742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:44.538742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:44.538742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:44.538742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:44.538742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:44.538742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:44.538742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:44.538742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:44.539674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:44.540547 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:44.540547 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:44.540547 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:44.540547 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:44.547045 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 307/597] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:44.549010 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00099b-6379-4d8a-acc4-84d2a154bc71 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:44.559986 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00126b-1794-48b3-8e6f-15050c9400ee None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:44.599474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64322720-85af-4e87-abcd-da58c44dfb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:44.600498 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64322720-85af-4e87-abcd-da58c44dfb7d None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:44.600705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64322720-85af-4e87-abcd-da58c44dfb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:44.601010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64322720-85af-4e87-abcd-da58c44dfb7d None admin] PATH_INFO: `/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/7fbf33189aa94f4f973d558d615a23e6/roles/2c276d442f7f4dd59ae7fc3a03943716` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:44.617501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64322720-85af-4e87-abcd-da58c44dfb7d None admin] RBAC: Authorizing `identity:create_grant(project_id=db47f1bb607c4d1a9c346fdc80ecadfa, user_id=7fbf33189aa94f4f973d558d615a23e6, role_id=2c276d442f7f4dd59ae7fc3a03943716)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:44.625049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64322720-85af-4e87-abcd-da58c44dfb7d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:44.675924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00099b-6379-4d8a-acc4-84d2a154bc71 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:44.697075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 291/598] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:02:44 2022] PUT /identity/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/7fbf33189aa94f4f973d558d615a23e6/roles/2c276d442f7f4dd59ae7fc3a03943716 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:02:44.698387 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64322720-85af-4e87-abcd-da58c44dfb7d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:44.727810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:44.731992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:44.738990 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e09d5-0798-4d33-af42-11b47bfac23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:44.740151 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e09d5-0798-4d33-af42-11b47bfac23e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:44.740615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e09d5-0798-4d33-af42-11b47bfac23e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:44.741054 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e09d5-0798-4d33-af42-11b47bfac23e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:44.763164 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e09d5-0798-4d33-af42-11b47bfac23e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:44.765070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e09d5-0798-4d33-af42-11b47bfac23e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:44.802682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-800e09d5-0798-4d33-af42-11b47bfac23e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:44.802682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:44.802682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:44.802682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:44.802682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:44.802682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:44.802682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:44.802682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:44.802682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:44.802682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:44.802682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:44.802682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:44.802682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:44.802682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:44.802682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:44.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:44.804854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:44.804854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:44.804854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:44.804854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:44.813364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 308/599] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:44.820504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e09d5-0798-4d33-af42-11b47bfac23e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:44.846773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64322720-85af-4e87-abcd-da58c44dfb7d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:44.894086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcafb3-7acb-483a-be8d-6f207558dd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:44.895024 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcafb3-7acb-483a-be8d-6f207558dd2b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:44.895409 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcafb3-7acb-483a-be8d-6f207558dd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:44.895788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcafb3-7acb-483a-be8d-6f207558dd2b None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:44.896670 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcafb3-7acb-483a-be8d-6f207558dd2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:44.900356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcafb3-7acb-483a-be8d-6f207558dd2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:44.900356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fcafb3-7acb-483a-be8d-6f207558dd2b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:44.914378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 292/600] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:44.922706 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcafb3-7acb-483a-be8d-6f207558dd2b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:44.991734 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e09d5-0798-4d33-af42-11b47bfac23e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:45.035877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812489c3-0936-4bcd-b67e-f0f37343cda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:45.036913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812489c3-0936-4bcd-b67e-f0f37343cda3 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:45.037532 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812489c3-0936-4bcd-b67e-f0f37343cda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:45.038012 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812489c3-0936-4bcd-b67e-f0f37343cda3 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:45.038963 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812489c3-0936-4bcd-b67e-f0f37343cda3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:45.039490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812489c3-0936-4bcd-b67e-f0f37343cda3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:45.041199 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812489c3-0936-4bcd-b67e-f0f37343cda3 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:45.054834 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 309/601] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:45.063380 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812489c3-0936-4bcd-b67e-f0f37343cda3 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:45.104217 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:45.120793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:45.172620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcafb3-7acb-483a-be8d-6f207558dd2b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:45.191145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812489c3-0936-4bcd-b67e-f0f37343cda3 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:45.245015 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ec48d-b4fc-42d0-8911-1441b5422b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:45.246121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ec48d-b4fc-42d0-8911-1441b5422b73 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:45.246609 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ec48d-b4fc-42d0-8911-1441b5422b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:45.247056 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ec48d-b4fc-42d0-8911-1441b5422b73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:45.256530 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ec48d-b4fc-42d0-8911-1441b5422b73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:45.258118 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ec48d-b4fc-42d0-8911-1441b5422b73 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:45.273700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe30ee-16f3-4bae-b698-e1f03085ce7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:45.273929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-9d7ec48d-b4fc-42d0-8911-1441b5422b73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:45.273929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:45.273929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:45.273929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:45.273929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:45.273929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:45.273929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:45.273929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:45.273929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:45.273929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:45.273929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:45.273929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:45.273929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:45.273929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:45.273929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:45.274781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:45.275659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:45.275659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:45.275659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:45.275659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:45.275659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:45.275659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:45.275659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:45.275659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:45.275659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 293/602] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:45.277093 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe30ee-16f3-4bae-b698-e1f03085ce7f None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:45.277590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe30ee-16f3-4bae-b698-e1f03085ce7f None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:45.278063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fe30ee-16f3-4bae-b698-e1f03085ce7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:45.286016 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ec48d-b4fc-42d0-8911-1441b5422b73 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:45.286561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe30ee-16f3-4bae-b698-e1f03085ce7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:45.288340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fe30ee-16f3-4bae-b698-e1f03085ce7f None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:45.307159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-c3fe30ee-16f3-4bae-b698-e1f03085ce7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:45.307159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:45.307159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:45.307159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:45.307159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:45.307159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:45.307159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:45.307159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:45.307159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:45.307159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:45.307159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:45.307159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:45.307159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:45.307159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:45.307159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:45.308106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:45.309001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:45.309001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:45.309001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:45.309001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:45.309001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:45.309001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:45.309001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:45.309001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:45.312218 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 310/603] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:45.317964 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe30ee-16f3-4bae-b698-e1f03085ce7f None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:45.400677 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ec48d-b4fc-42d0-8911-1441b5422b73 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:45.445165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d57d7-cdd8-4783-85ad-b06069408873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:45.445776 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d57d7-cdd8-4783-85ad-b06069408873 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:45.446057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d57d7-cdd8-4783-85ad-b06069408873 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:45.446334 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d57d7-cdd8-4783-85ad-b06069408873 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:45.447160 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d57d7-cdd8-4783-85ad-b06069408873 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:45.447479 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d57d7-cdd8-4783-85ad-b06069408873 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:45.448927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d57d7-cdd8-4783-85ad-b06069408873 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:45.466322 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:45.477298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 294/604] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:45.487096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d57d7-cdd8-4783-85ad-b06069408873 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:45.522221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fe30ee-16f3-4bae-b698-e1f03085ce7f None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:45.531638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:45.576379 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ddf89-4bc4-468a-b4a6-30b431be589b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:45.577647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ddf89-4bc4-468a-b4a6-30b431be589b None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:45.578026 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ddf89-4bc4-468a-b4a6-30b431be589b None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:45.578314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ddf89-4bc4-468a-b4a6-30b431be589b None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:45.579128 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ddf89-4bc4-468a-b4a6-30b431be589b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:45.579444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ddf89-4bc4-468a-b4a6-30b431be589b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:45.581712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ddf89-4bc4-468a-b4a6-30b431be589b None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:45.596890 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 311/605] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:45.603493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ddf89-4bc4-468a-b4a6-30b431be589b None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:45.660656 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d57d7-cdd8-4783-85ad-b06069408873 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:45.696971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec6b3a-f246-4336-b206-8ce86e87d88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:45.697798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec6b3a-f246-4336-b206-8ce86e87d88a None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:45.698079 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec6b3a-f246-4336-b206-8ce86e87d88a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:45.698355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec6b3a-f246-4336-b206-8ce86e87d88a None admin] PATH_INFO: `/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/ae926ce83d464c768baddd90de31aa40/roles/2c276d442f7f4dd59ae7fc3a03943716` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:45.715032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec6b3a-f246-4336-b206-8ce86e87d88a None admin] RBAC: Authorizing `identity:create_grant(project_id=db47f1bb607c4d1a9c346fdc80ecadfa, user_id=ae926ce83d464c768baddd90de31aa40, role_id=2c276d442f7f4dd59ae7fc3a03943716)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:45.716530 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ec6b3a-f246-4336-b206-8ce86e87d88a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:45.783385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ddf89-4bc4-468a-b4a6-30b431be589b None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:45.817669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 295/606] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:02:45 2022] PUT /identity/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/ae926ce83d464c768baddd90de31aa40/roles/2c276d442f7f4dd59ae7fc3a03943716 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:02:45.835575 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1270e99-4c96-47ce-992a-5cbd5e28cd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:45.836611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1270e99-4c96-47ce-992a-5cbd5e28cd4b None admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:45.836900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1270e99-4c96-47ce-992a-5cbd5e28cd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:45.837209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1270e99-4c96-47ce-992a-5cbd5e28cd4b None admin] PATH_INFO: `/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/fa51162a69d44127bd34df2f4653dc15/roles/2c276d442f7f4dd59ae7fc3a03943716` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:45.856189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1270e99-4c96-47ce-992a-5cbd5e28cd4b None admin] RBAC: Authorizing `identity:create_grant(project_id=db47f1bb607c4d1a9c346fdc80ecadfa, user_id=fa51162a69d44127bd34df2f4653dc15, role_id=2c276d442f7f4dd59ae7fc3a03943716)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:45.858496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1270e99-4c96-47ce-992a-5cbd5e28cd4b None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:45.899923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b423b43-7dbb-472e-88cd-f36484d386ed None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:45.900318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b423b43-7dbb-472e-88cd-f36484d386ed None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:45.900457 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b423b43-7dbb-472e-88cd-f36484d386ed None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:45.902272 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 296/607] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:45.908812 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e66dd-7225-4153-9e9b-108588d04d8d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:45.909207 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e66dd-7225-4153-9e9b-108588d04d8d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:45.909429 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e66dd-7225-4153-9e9b-108588d04d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:45.928738 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-6b3e66dd-7225-4153-9e9b-108588d04d8d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:45.930001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-6b3e66dd-7225-4153-9e9b-108588d04d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:45.940804 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 312/608] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:02:45 2022] PUT /identity/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/fa51162a69d44127bd34df2f4653dc15/roles/2c276d442f7f4dd59ae7fc3a03943716 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:02:46.000467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:46.040337 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 297/609] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:46.065191 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1270e99-4c96-47ce-992a-5cbd5e28cd4b None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:46.085617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:46.193010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f85df-2143-42c6-ab32-490ec2f862b5 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:46.204365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f85df-2143-42c6-ab32-490ec2f862b5 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:46.204365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f85df-2143-42c6-ab32-490ec2f862b5 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:46.204365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 298/610] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:46.204365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d1743-1f0e-43f1-9f9d-3a6314b2e3b7 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:46.205332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1270e99-4c96-47ce-992a-5cbd5e28cd4b None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:46.205646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d1743-1f0e-43f1-9f9d-3a6314b2e3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:46.205646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d1743-1f0e-43f1-9f9d-3a6314b2e3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:46.225841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a41d1743-1f0e-43f1-9f9d-3a6314b2e3b7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:46.227197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a41d1743-1f0e-43f1-9f9d-3a6314b2e3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:46.254571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211f576-d050-43cf-ac0d-923cc7caaa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:46.255353 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211f576-d050-43cf-ac0d-923cc7caaa52 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:46.257205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211f576-d050-43cf-ac0d-923cc7caaa52 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:46.258402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211f576-d050-43cf-ac0d-923cc7caaa52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:46.273890 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211f576-d050-43cf-ac0d-923cc7caaa52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-a211f576-d050-43cf-ac0d-923cc7caaa52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:46.288187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:46.289240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:46.290141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:46.290141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:46.290141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:46.290141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:46.290141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:46.290141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:46.290141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:46.290141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:46.290141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:46.290141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:46.290141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:46.290141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:46.292435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 313/611] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:46.303545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211f576-d050-43cf-ac0d-923cc7caaa52 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:46.328729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 299/612] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:02:46.353941 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d1743-1f0e-43f1-9f9d-3a6314b2e3b7 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:46.439982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:46.492125 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d1743-1f0e-43f1-9f9d-3a6314b2e3b7 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:46.495631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211f576-d050-43cf-ac0d-923cc7caaa52 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:46.511298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:46.548289 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2937678-fba8-427c-8afe-d8bb53f27d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:46.548992 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2937678-fba8-427c-8afe-d8bb53f27d6d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:46.549285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2937678-fba8-427c-8afe-d8bb53f27d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:46.549558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2937678-fba8-427c-8afe-d8bb53f27d6d None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:46.550328 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2937678-fba8-427c-8afe-d8bb53f27d6d None admin] RBAC: Adding query filter params (name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:46.550871 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2937678-fba8-427c-8afe-d8bb53f27d6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:46.552261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2937678-fba8-427c-8afe-d8bb53f27d6d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:46.563744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a69433-fb36-43ad-8253-56dd3e184a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:46.564556 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a69433-fb36-43ad-8253-56dd3e184a1b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:46.564835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a69433-fb36-43ad-8253-56dd3e184a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:46.565143 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a69433-fb36-43ad-8253-56dd3e184a1b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:46.570903 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 314/613] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:46.578373 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2937678-fba8-427c-8afe-d8bb53f27d6d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:46.592889 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a69433-fb36-43ad-8253-56dd3e184a1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-66a69433-fb36-43ad-8253-56dd3e184a1b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:46.602392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:46.603395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:46.604194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:46.604194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:46.604194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:46.604194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:46.604194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:46.604194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:46.604194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:46.604194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:46.604194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:46.604194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:46.604194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:46.604194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:46.606637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 300/614] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:46.610810 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a69433-fb36-43ad-8253-56dd3e184a1b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:46.790931 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2937678-fba8-427c-8afe-d8bb53f27d6d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:46.814167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a69433-fb36-43ad-8253-56dd3e184a1b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:46.857079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f85ae-2eb9-4f46-9b59-af0a8bf7afe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:46.858445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f85ae-2eb9-4f46-9b59-af0a8bf7afe2 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:46.858733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f85ae-2eb9-4f46-9b59-af0a8bf7afe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:46.859015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f85ae-2eb9-4f46-9b59-af0a8bf7afe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:46.866298 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:46.879717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f85ae-2eb9-4f46-9b59-af0a8bf7afe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:46.881220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f85ae-2eb9-4f46-9b59-af0a8bf7afe2 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:46.883131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26617292-ce6a-4598-8d68-7ef044f5c754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:46.883809 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26617292-ce6a-4598-8d68-7ef044f5c754 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:46.884090 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26617292-ce6a-4598-8d68-7ef044f5c754 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:46.884367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26617292-ce6a-4598-8d68-7ef044f5c754 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:46.885156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26617292-ce6a-4598-8d68-7ef044f5c754 None admin] RBAC: Adding query filter params (name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:46.885472 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26617292-ce6a-4598-8d68-7ef044f5c754 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:46.886928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26617292-ce6a-4598-8d68-7ef044f5c754 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:46.903444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 301/615] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:46.909281 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26617292-ce6a-4598-8d68-7ef044f5c754 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:46.911136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-267f85ae-2eb9-4f46-9b59-af0a8bf7afe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:46.911136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:46.911136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:46.911136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:46.911136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:46.911136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:46.911136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:46.911136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:46.911136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:46.911136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:46.911136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:46.911136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:46.911136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:46.911136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:46.911136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:46.911994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:46.912854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:46.912854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:46.912854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:46.912854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:46.912854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 315/616] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:46.921530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f85ae-2eb9-4f46-9b59-af0a8bf7afe2 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:46.950098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:47.041071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26617292-ce6a-4598-8d68-7ef044f5c754 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:47.113695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb19b9-67bc-4452-b759-11480eddd8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:47.114973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb19b9-67bc-4452-b759-11480eddd8ec None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:47.116916 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb19b9-67bc-4452-b759-11480eddd8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:47.118736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb19b9-67bc-4452-b759-11480eddd8ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:47.143833 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb19b9-67bc-4452-b759-11480eddd8ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:47.146671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb19b9-67bc-4452-b759-11480eddd8ec None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:47.159149 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f85ae-2eb9-4f46-9b59-af0a8bf7afe2 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:47.183214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-afdb19b9-67bc-4452-b759-11480eddd8ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:47.183214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:47.183214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:47.183214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:47.183214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:47.183214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:47.183214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:47.183214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:47.183214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:47.183214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:47.183214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:47.183214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:47.183214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:47.183214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:47.183214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:47.184158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:47.185037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:47.185037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:47.185037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:47.185037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:47.188874 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 302/617] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:47.194872 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb19b9-67bc-4452-b759-11480eddd8ec None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:47.241961 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b76776-0896-418c-9284-7da930a0002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:47.243761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b76776-0896-418c-9284-7da930a0002c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:47.244447 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b76776-0896-418c-9284-7da930a0002c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:47.245919 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b76776-0896-418c-9284-7da930a0002c None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:47.246880 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b76776-0896-418c-9284-7da930a0002c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:47.247550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b76776-0896-418c-9284-7da930a0002c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:47.250464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b76776-0896-418c-9284-7da930a0002c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:47.277528 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 316/618] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:47.280779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b76776-0896-418c-9284-7da930a0002c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:47.351610 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:47.388595 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb19b9-67bc-4452-b759-11480eddd8ec None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:47.394567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:47.448313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e77cf-bcfe-40c1-8da1-55c4187b51f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:47.449410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e77cf-bcfe-40c1-8da1-55c4187b51f3 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:47.449904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e77cf-bcfe-40c1-8da1-55c4187b51f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:47.450381 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e77cf-bcfe-40c1-8da1-55c4187b51f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:47.451314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e77cf-bcfe-40c1-8da1-55c4187b51f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:47.451999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e77cf-bcfe-40c1-8da1-55c4187b51f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:47.453771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e77cf-bcfe-40c1-8da1-55c4187b51f3 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:47.473865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 303/619] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:47.479375 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e77cf-bcfe-40c1-8da1-55c4187b51f3 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:47.510522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b76776-0896-418c-9284-7da930a0002c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:47.607393 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3168cde-bcc9-4cec-aa2e-5e27e676afd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:47.609197 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3168cde-bcc9-4cec-aa2e-5e27e676afd1 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:47.609741 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3168cde-bcc9-4cec-aa2e-5e27e676afd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:47.610244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3168cde-bcc9-4cec-aa2e-5e27e676afd1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:47.625296 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3168cde-bcc9-4cec-aa2e-5e27e676afd1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:47.633768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3168cde-bcc9-4cec-aa2e-5e27e676afd1 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:47.656988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e77cf-bcfe-40c1-8da1-55c4187b51f3 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-d3168cde-bcc9-4cec-aa2e-5e27e676afd1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:47.667270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:47.668284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:47.669251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:47.669251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:47.669251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:47.669251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:47.669251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:47.669251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:47.669251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:47.669251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:47.669251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:02:47.669251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:47.672589 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 317/620] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:02:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:47.678122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3168cde-bcc9-4cec-aa2e-5e27e676afd1 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:47.705894 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649ee0a-0e05-428b-a9d1-f33e408cf883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:47.706675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649ee0a-0e05-428b-a9d1-f33e408cf883 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:47.706966 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649ee0a-0e05-428b-a9d1-f33e408cf883 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:47.707246 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649ee0a-0e05-428b-a9d1-f33e408cf883 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:47.715470 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649ee0a-0e05-428b-a9d1-f33e408cf883 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:47.717011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649ee0a-0e05-428b-a9d1-f33e408cf883 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-c649ee0a-0e05-428b-a9d1-f33e408cf883 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:47.744521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:47.745573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:47.746590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:47.746590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:47.746590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:47.746590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:47.746590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:47.746590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:47.746590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:47.746590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:47.746590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:02:47.746590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:47.751293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 304/621] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jun 15 10:02:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:02:47.752730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1617b-67b8-49bc-8d72-f8afcce29c28 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:47.753112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1617b-67b8-49bc-8d72-f8afcce29c28 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:47.753333 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1617b-67b8-49bc-8d72-f8afcce29c28 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:47.754971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 305/622] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:47.755678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1617b-67b8-49bc-8d72-f8afcce29c28 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:47.762526 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:47.811720 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3168cde-bcc9-4cec-aa2e-5e27e676afd1 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:47.822476 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:47.849986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab75c2e-6dd5-4201-a1a9-e9ee0f483a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:47.851111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab75c2e-6dd5-4201-a1a9-e9ee0f483a78 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:47.851672 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab75c2e-6dd5-4201-a1a9-e9ee0f483a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:47.852229 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab75c2e-6dd5-4201-a1a9-e9ee0f483a78 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:47.853737 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab75c2e-6dd5-4201-a1a9-e9ee0f483a78 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:47.854291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab75c2e-6dd5-4201-a1a9-e9ee0f483a78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:47.856303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab75c2e-6dd5-4201-a1a9-e9ee0f483a78 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:47.874921 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1617b-67b8-49bc-8d72-f8afcce29c28 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:47.889703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 318/623] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Jun 15 10:02:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:47.892490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b61e6c-41ec-4abc-96d9-3bbf53395add None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:47.893335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b61e6c-41ec-4abc-96d9-3bbf53395add None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:47.894121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b61e6c-41ec-4abc-96d9-3bbf53395add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:47.914482 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252415b-bf26-4aea-b0ac-9668a96bde5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:47.914985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-77b61e6c-41ec-4abc-96d9-3bbf53395add None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:47.915221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252415b-bf26-4aea-b0ac-9668a96bde5b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:47.915509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252415b-bf26-4aea-b0ac-9668a96bde5b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:47.915788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252415b-bf26-4aea-b0ac-9668a96bde5b None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:47.916707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252415b-bf26-4aea-b0ac-9668a96bde5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:47.917065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252415b-bf26-4aea-b0ac-9668a96bde5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:47.917537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-77b61e6c-41ec-4abc-96d9-3bbf53395add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:47.918564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252415b-bf26-4aea-b0ac-9668a96bde5b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:47.944848 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 306/624] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jun 15 10:02:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:47.945736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252415b-bf26-4aea-b0ac-9668a96bde5b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:47.991022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 319/625] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:47.992281 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b61e6c-41ec-4abc-96d9-3bbf53395add None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:48.033559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:48.085880 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252415b-bf26-4aea-b0ac-9668a96bde5b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:48.103216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:48.110169 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b61e6c-41ec-4abc-96d9-3bbf53395add None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:48.124515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dde5dd-ccb2-4429-bd54-0d4b3f455c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:48.125499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dde5dd-ccb2-4429-bd54-0d4b3f455c85 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:48.125730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dde5dd-ccb2-4429-bd54-0d4b3f455c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:48.126022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dde5dd-ccb2-4429-bd54-0d4b3f455c85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:48.138487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dde5dd-ccb2-4429-bd54-0d4b3f455c85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:48.140607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dde5dd-ccb2-4429-bd54-0d4b3f455c85 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:48.150938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06eea81-5d38-42ce-a525-f6994b984fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:48.151944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06eea81-5d38-42ce-a525-f6994b984fbf None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:48.152514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06eea81-5d38-42ce-a525-f6994b984fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:48.153074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06eea81-5d38-42ce-a525-f6994b984fbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:48.161815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-35dde5dd-ccb2-4429-bd54-0d4b3f455c85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:48.161815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:48.161815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:48.161815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:48.161815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:48.161815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:48.161815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:48.161815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:48.161815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:48.161815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:48.161815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:48.161815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:48.161815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:48.161815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:48.161815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:48.162753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:48.163642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:48.163642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:48.163642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:48.163642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:48.163642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 307/626] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:48.164367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dde5dd-ccb2-4429-bd54-0d4b3f455c85 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:48.234341 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06eea81-5d38-42ce-a525-f6994b984fbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:48.236553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06eea81-5d38-42ce-a525-f6994b984fbf None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:48.279891 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dde5dd-ccb2-4429-bd54-0d4b3f455c85 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:48.294978 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:48.319264 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afe835-52d4-47f2-838c-0c2a544b7273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:48.320270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afe835-52d4-47f2-838c-0c2a544b7273 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:48.320451 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afe835-52d4-47f2-838c-0c2a544b7273 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:48.320737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afe835-52d4-47f2-838c-0c2a544b7273 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:48.333099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afe835-52d4-47f2-838c-0c2a544b7273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:48.338520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-a5afe835-52d4-47f2-838c-0c2a544b7273 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:48.344528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:48.345647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:48.346481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:48.346481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:48.346481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:48.346481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:48.346481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:48.346481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:48.346481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:48.346481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:48.346481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:48.346481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:48.346481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:48.346481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:48.353046 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 320/627] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:48.353309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06eea81-5d38-42ce-a525-f6994b984fbf None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:48.353708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 308/628] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:48.354768 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afe835-52d4-47f2-838c-0c2a544b7273 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:48.461368 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06eea81-5d38-42ce-a525-f6994b984fbf None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:48.470604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afe835-52d4-47f2-838c-0c2a544b7273 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:48.507259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141a0cb-c5b2-4333-9308-fd81b33f94ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:48.508060 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141a0cb-c5b2-4333-9308-fd81b33f94ca None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:48.508346 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141a0cb-c5b2-4333-9308-fd81b33f94ca None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:48.508636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141a0cb-c5b2-4333-9308-fd81b33f94ca None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:48.509653 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141a0cb-c5b2-4333-9308-fd81b33f94ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:48.509969 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141a0cb-c5b2-4333-9308-fd81b33f94ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:48.511646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141a0cb-c5b2-4333-9308-fd81b33f94ca None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:48.512286 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b078c-4401-4a5d-90cc-3fb4db9ad83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:48.513480 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b078c-4401-4a5d-90cc-3fb4db9ad83b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:48.514702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b078c-4401-4a5d-90cc-3fb4db9ad83b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:48.515204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b078c-4401-4a5d-90cc-3fb4db9ad83b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:48.533614 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b078c-4401-4a5d-90cc-3fb4db9ad83b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:48.535620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b078c-4401-4a5d-90cc-3fb4db9ad83b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:48.539188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 321/629] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:48.539188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141a0cb-c5b2-4333-9308-fd81b33f94ca None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:48.563159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-2d3b078c-4401-4a5d-90cc-3fb4db9ad83b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:48.563159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:48.563159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:48.563159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:48.563159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:48.563159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:48.563159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:48.563159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:48.563159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:48.563159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:48.563159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:48.563159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:48.563159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:48.563159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:48.563159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:48.564156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:48.565057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:48.565057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:48.565057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:48.565057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:48.567902 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 309/630] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:48.569068 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b078c-4401-4a5d-90cc-3fb4db9ad83b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:48.598881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:48.612682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:48.655770 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141a0cb-c5b2-4333-9308-fd81b33f94ca None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:48.676434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b078c-4401-4a5d-90cc-3fb4db9ad83b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:48.690193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37b575-1101-42c9-b19a-40a9222ec20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:48.690857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37b575-1101-42c9-b19a-40a9222ec20f None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:48.691142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37b575-1101-42c9-b19a-40a9222ec20f None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:48.691417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37b575-1101-42c9-b19a-40a9222ec20f None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:48.692194 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37b575-1101-42c9-b19a-40a9222ec20f None admin] RBAC: Adding query filter params (name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:48.692501 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37b575-1101-42c9-b19a-40a9222ec20f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:48.693925 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37b575-1101-42c9-b19a-40a9222ec20f None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:48.702254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 322/631] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:48.706771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37b575-1101-42c9-b19a-40a9222ec20f None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:48.710275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a2921-60b0-4bc0-98fe-b8b8e25f8be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:48.711074 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a2921-60b0-4bc0-98fe-b8b8e25f8be6 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:48.711356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a2921-60b0-4bc0-98fe-b8b8e25f8be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:48.711681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a2921-60b0-4bc0-98fe-b8b8e25f8be6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:48.718995 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a2921-60b0-4bc0-98fe-b8b8e25f8be6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:48.720436 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a2921-60b0-4bc0-98fe-b8b8e25f8be6 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:48.734633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-1d6a2921-60b0-4bc0-98fe-b8b8e25f8be6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:48.734633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:48.734633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:48.734633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:48.734633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:48.734633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:48.734633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:48.734633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:48.734633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:48.734633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:48.734633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:48.734633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:48.734633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:48.734633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:48.734633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:48.735717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:48.736762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:48.736762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:48.736762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:48.736762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:48.736762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:48.736762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:48.736762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:48.736762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:48.736762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 310/632] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:48.737388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785c98e-5e7f-4f89-8836-9aa1975fdd1b None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:48.737388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785c98e-5e7f-4f89-8836-9aa1975fdd1b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:48.737598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785c98e-5e7f-4f89-8836-9aa1975fdd1b None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:48.741023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 311/633] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:48.741023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20551f2-c193-43b6-be16-392a8fdb4c7b None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:48.741023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20551f2-c193-43b6-be16-392a8fdb4c7b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:48.741023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20551f2-c193-43b6-be16-392a8fdb4c7b None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:48.741971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 312/634] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:48.742660 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20551f2-c193-43b6-be16-392a8fdb4c7b None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:48.815532 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37b575-1101-42c9-b19a-40a9222ec20f None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:48.832624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:48.833521 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:48.853991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dc2cb-7a8b-443f-b43b-51932f7caf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:48.854665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dc2cb-7a8b-443f-b43b-51932f7caf9f None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:48.854949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dc2cb-7a8b-443f-b43b-51932f7caf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:48.855117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20551f2-c193-43b6-be16-392a8fdb4c7b None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:48.855214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dc2cb-7a8b-443f-b43b-51932f7caf9f None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:48.855972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9dc2cb-7a8b-443f-b43b-51932f7caf9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:48.856280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9dc2cb-7a8b-443f-b43b-51932f7caf9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:48.858682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9dc2cb-7a8b-443f-b43b-51932f7caf9f None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:48.871350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 323/635] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:48.872206 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dc2cb-7a8b-443f-b43b-51932f7caf9f None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:48.891305 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1042520-cc70-427a-bfd8-19e6640c8021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:48.892107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1042520-cc70-427a-bfd8-19e6640c8021 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:48.892397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1042520-cc70-427a-bfd8-19e6640c8021 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:48.892769 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1042520-cc70-427a-bfd8-19e6640c8021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:48.903031 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1042520-cc70-427a-bfd8-19e6640c8021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:48.905791 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1042520-cc70-427a-bfd8-19e6640c8021 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:48.923293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-f1042520-cc70-427a-bfd8-19e6640c8021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:48.923293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:48.923293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:48.923293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:48.923293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:48.923293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:48.923293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:48.923293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:48.923293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:48.923293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:48.923293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:48.923293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:48.923293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:48.923293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:48.923293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:48.924397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:48.925825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:48.925825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:48.925825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:48.925825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:48.925825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 313/636] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:48.926183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535041bf-2ef2-475e-a34f-e34a86fda7b8 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:48.926272 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535041bf-2ef2-475e-a34f-e34a86fda7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:48.926805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535041bf-2ef2-475e-a34f-e34a86fda7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:48.946966 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-535041bf-2ef2-475e-a34f-e34a86fda7b8 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:48.948294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-535041bf-2ef2-475e-a34f-e34a86fda7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:48.982276 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dc2cb-7a8b-443f-b43b-51932f7caf9f None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:49.016644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6567f2b-cb6e-4414-b976-decaf817ce16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:49.017338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6567f2b-cb6e-4414-b976-decaf817ce16 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:49.017622 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6567f2b-cb6e-4414-b976-decaf817ce16 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:49.017900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6567f2b-cb6e-4414-b976-decaf817ce16 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:49.018683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6567f2b-cb6e-4414-b976-decaf817ce16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:49.018991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6567f2b-cb6e-4414-b976-decaf817ce16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:49.019435 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 314/637] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:49.020429 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6567f2b-cb6e-4414-b976-decaf817ce16 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:49.020799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb0d67-46e7-417f-9992-2e4673601c92 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:49.021126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb0d67-46e7-417f-9992-2e4673601c92 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:49.021404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb0d67-46e7-417f-9992-2e4673601c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:49.038134 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-f8bb0d67-46e7-417f-9992-2e4673601c92 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:49.039338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f8bb0d67-46e7-417f-9992-2e4673601c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:49.040759 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 324/638] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:49.046780 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6567f2b-cb6e-4414-b976-decaf817ce16 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:49.086908 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:49.100973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:49.108856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 315/639] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:49.109738 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb0d67-46e7-417f-9992-2e4673601c92 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:49.157990 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6567f2b-cb6e-4414-b976-decaf817ce16 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:49.201128 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d2b27-38fd-45cd-bd33-1565c35b39ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:49.203132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d2b27-38fd-45cd-bd33-1565c35b39ec None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:49.203414 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d2b27-38fd-45cd-bd33-1565c35b39ec None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:49.203689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d2b27-38fd-45cd-bd33-1565c35b39ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:49.211485 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d2b27-38fd-45cd-bd33-1565c35b39ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:49.212949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d2b27-38fd-45cd-bd33-1565c35b39ec None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:49.229629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-479d2b27-38fd-45cd-bd33-1565c35b39ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:49.229629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:49.229629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:49.229629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:49.229629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:49.229629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:49.229629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:49.229629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:49.229629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:49.229629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:49.229629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:49.229629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:49.229629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:49.229629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:49.229629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:49.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:49.231569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:49.231569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:49.231569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:49.231569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:49.231569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:49.231569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:49.231569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:49.231569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:49.231990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb0d67-46e7-417f-9992-2e4673601c92 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:49.232071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 325/640] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:49.232071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d2b27-38fd-45cd-bd33-1565c35b39ec None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:49.270919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d314e2-ba06-499d-9550-fca6d0f9947c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:49.271505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d314e2-ba06-499d-9550-fca6d0f9947c None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:49.271787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d314e2-ba06-499d-9550-fca6d0f9947c None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:49.272063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d314e2-ba06-499d-9550-fca6d0f9947c None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:49.272786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d314e2-ba06-499d-9550-fca6d0f9947c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:49.273123 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d314e2-ba06-499d-9550-fca6d0f9947c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:49.274493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d314e2-ba06-499d-9550-fca6d0f9947c None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:49.286691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 316/641] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:49.287465 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d314e2-ba06-499d-9550-fca6d0f9947c None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:49.328850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:49.333279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d2b27-38fd-45cd-bd33-1565c35b39ec None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:49.347506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:49.368304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd2519-35cd-416d-9b04-5b97b53d26c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:49.369204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd2519-35cd-416d-9b04-5b97b53d26c2 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:49.369490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd2519-35cd-416d-9b04-5b97b53d26c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:49.369768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd2519-35cd-416d-9b04-5b97b53d26c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:49.379863 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd2519-35cd-416d-9b04-5b97b53d26c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-6bbd2519-35cd-416d-9b04-5b97b53d26c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:49.387649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:49.388650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:49.390549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:49.390549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:49.390549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:49.390549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:49.390549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:49.390549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:49.390549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:49.390549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:49.390549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:49.390549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:49.390549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:49.390549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:49.390549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 326/642] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:49.390549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd2519-35cd-416d-9b04-5b97b53d26c2 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:49.396579 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d314e2-ba06-499d-9550-fca6d0f9947c None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:49.434442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bd191-6518-4b0e-808d-5b6dfd495805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:49.435330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bd191-6518-4b0e-808d-5b6dfd495805 None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:49.435618 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bd191-6518-4b0e-808d-5b6dfd495805 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:49.436241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bd191-6518-4b0e-808d-5b6dfd495805 None admin] PATH_INFO: `/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/589e66c6228a44adbc9e418794d7d91b/roles/2c276d442f7f4dd59ae7fc3a03943716` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:49.450263 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7bd191-6518-4b0e-808d-5b6dfd495805 None admin] RBAC: Authorizing `identity:create_grant(project_id=db47f1bb607c4d1a9c346fdc80ecadfa, user_id=589e66c6228a44adbc9e418794d7d91b, role_id=2c276d442f7f4dd59ae7fc3a03943716)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:49.452416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7bd191-6518-4b0e-808d-5b6dfd495805 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:49.498007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd2519-35cd-416d-9b04-5b97b53d26c2 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:49.535077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadb378-107a-47e2-9d08-0aa41d912a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:49.536247 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadb378-107a-47e2-9d08-0aa41d912a3e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:49.536710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadb378-107a-47e2-9d08-0aa41d912a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:49.537176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 317/643] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:02:49 2022] PUT /identity/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/589e66c6228a44adbc9e418794d7d91b/roles/2c276d442f7f4dd59ae7fc3a03943716 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:02:49.539237 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbadb378-107a-47e2-9d08-0aa41d912a3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:49.539339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bd191-6518-4b0e-808d-5b6dfd495805 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:49.561875 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbadb378-107a-47e2-9d08-0aa41d912a3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-cbadb378-107a-47e2-9d08-0aa41d912a3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:49.569953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:49.572129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:49.574261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:49.574261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:49.574261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:49.574261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:49.574261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:49.574261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:49.574261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:49.574261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:49.574261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:49.574261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:49.574261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:49.574261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:49.574261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 327/644] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:49.574261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadb378-107a-47e2-9d08-0aa41d912a3e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:49.597928 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:49.680584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bd191-6518-4b0e-808d-5b6dfd495805 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:49.690345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:49.697625 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbadb378-107a-47e2-9d08-0aa41d912a3e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:49.727183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c2291-3cd6-490d-b681-850583796637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:49.727854 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c2291-3cd6-490d-b681-850583796637 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:49.728144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c2291-3cd6-490d-b681-850583796637 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:49.728416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c2291-3cd6-490d-b681-850583796637 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:49.729291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c2291-3cd6-490d-b681-850583796637 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:49.729602 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c2291-3cd6-490d-b681-850583796637 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:49.731130 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c2291-3cd6-490d-b681-850583796637 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:49.740524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00759cf-75a5-4016-b6e1-6cefa39423b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:49.741435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00759cf-75a5-4016-b6e1-6cefa39423b2 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:49.741717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00759cf-75a5-4016-b6e1-6cefa39423b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:49.741997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00759cf-75a5-4016-b6e1-6cefa39423b2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:49.744550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 318/645] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:49.745579 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c2291-3cd6-490d-b681-850583796637 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:49.754696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00759cf-75a5-4016-b6e1-6cefa39423b2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:49.756158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00759cf-75a5-4016-b6e1-6cefa39423b2 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-f00759cf-75a5-4016-b6e1-6cefa39423b2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:49.771096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:49.772076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:49.773663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:49.773663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:49.773663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:49.773663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:49.773663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:49.773663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:49.773663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:49.773663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:49.773663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 15 10:02:49.773663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:49.773663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 328/646] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:49.773663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00759cf-75a5-4016-b6e1-6cefa39423b2 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:49.856859 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c2291-3cd6-490d-b681-850583796637 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:49.865634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:49.893820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00759cf-75a5-4016-b6e1-6cefa39423b2 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:49.897340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf666e-57b6-4cd8-bbf9-171afb17996a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:49.898016 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf666e-57b6-4cd8-bbf9-171afb17996a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:49.898304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf666e-57b6-4cd8-bbf9-171afb17996a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:49.898580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf666e-57b6-4cd8-bbf9-171afb17996a None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:49.899338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf666e-57b6-4cd8-bbf9-171afb17996a None admin] RBAC: Adding query filter params (name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:49.899646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf666e-57b6-4cd8-bbf9-171afb17996a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:49.901047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf666e-57b6-4cd8-bbf9-171afb17996a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:49.910394 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 319/647] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:49.911241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf666e-57b6-4cd8-bbf9-171afb17996a None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:49.937925 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0a1d9-e7ce-4e3b-9e77-16857b990310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:49.938596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0a1d9-e7ce-4e3b-9e77-16857b990310 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:49.938882 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0a1d9-e7ce-4e3b-9e77-16857b990310 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:49.939185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0a1d9-e7ce-4e3b-9e77-16857b990310 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:49.939916 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0a1d9-e7ce-4e3b-9e77-16857b990310 None admin] RBAC: Adding query filter params (name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:49.940238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0a1d9-e7ce-4e3b-9e77-16857b990310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:49.941197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:49.941630 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0a1d9-e7ce-4e3b-9e77-16857b990310 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:49.955582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 329/648] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:49.955582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0a1d9-e7ce-4e3b-9e77-16857b990310 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:50.019160 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf666e-57b6-4cd8-bbf9-171afb17996a None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:50.054400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9197b9c-c580-477f-9a88-10e021db25a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:50.055069 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9197b9c-c580-477f-9a88-10e021db25a4 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:50.055353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9197b9c-c580-477f-9a88-10e021db25a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:50.055631 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9197b9c-c580-477f-9a88-10e021db25a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:50.056475 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9197b9c-c580-477f-9a88-10e021db25a4 None admin] RBAC: Adding query filter params (role.id=2c276d442f7f4dd59ae7fc3a03943716, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=26f23f8595fc43ad94aee7d0192164a1) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:50.056791 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9197b9c-c580-477f-9a88-10e021db25a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:50.058271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9197b9c-c580-477f-9a88-10e021db25a4 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:50.072987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 320/649] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:02:49 2022] GET /identity/v3/role_assignments?user.id=26f23f8595fc43ad94aee7d0192164a1&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=2c276d442f7f4dd59ae7fc3a03943716 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:50.072987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9197b9c-c580-477f-9a88-10e021db25a4 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:50.127665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0a1d9-e7ce-4e3b-9e77-16857b990310 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:50.186420 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359c61c-d350-4f66-b3bb-78107d54f130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:50.188084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359c61c-d350-4f66-b3bb-78107d54f130 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:50.188474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359c61c-d350-4f66-b3bb-78107d54f130 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:50.189126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359c61c-d350-4f66-b3bb-78107d54f130 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:50.190084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359c61c-d350-4f66-b3bb-78107d54f130 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:50.190713 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359c61c-d350-4f66-b3bb-78107d54f130 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:50.191768 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:50.194043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359c61c-d350-4f66-b3bb-78107d54f130 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:50.202175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9197b9c-c580-477f-9a88-10e021db25a4 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:50.208557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:50.220536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 330/650] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Jun 15 10:02:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:50.221955 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359c61c-d350-4f66-b3bb-78107d54f130 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:50.262408 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef527f5a-3d88-48ff-9f39-7448e26b71e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:50.263264 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef527f5a-3d88-48ff-9f39-7448e26b71e3 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:50.263568 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef527f5a-3d88-48ff-9f39-7448e26b71e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:50.263984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef527f5a-3d88-48ff-9f39-7448e26b71e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:50.278752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef527f5a-3d88-48ff-9f39-7448e26b71e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:50.281862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef527f5a-3d88-48ff-9f39-7448e26b71e3 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:50.312755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-ef527f5a-3d88-48ff-9f39-7448e26b71e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:50.312755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:50.312755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:50.312755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:50.312755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:50.312755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:50.312755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:50.312755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:50.312755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:50.312755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:50.312755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:50.312755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:50.312755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:50.312755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:50.312755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:50.314310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:50.315371 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:50.315371 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:50.315371 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:50.315371 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:50.316904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 321/651] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:50.317943 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef527f5a-3d88-48ff-9f39-7448e26b71e3 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:50.378179 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359c61c-d350-4f66-b3bb-78107d54f130 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:50.417181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e78687-2f2c-4851-b674-f19ad1209640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:50.417991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e78687-2f2c-4851-b674-f19ad1209640 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:50.418277 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e78687-2f2c-4851-b674-f19ad1209640 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:50.419136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e78687-2f2c-4851-b674-f19ad1209640 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:50.431508 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e78687-2f2c-4851-b674-f19ad1209640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:50.433003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e78687-2f2c-4851-b674-f19ad1209640 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:50.442022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef527f5a-3d88-48ff-9f39-7448e26b71e3 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:50.453099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-86e78687-2f2c-4851-b674-f19ad1209640 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:50.453099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:50.453099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:50.453099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:50.453099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:50.453099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:50.453099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:50.453099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:50.453099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:50.453099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:50.453099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:50.453099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:50.453099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:50.453099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:50.453099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:50.453998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:50.454849 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:50.454849 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:50.454849 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:50.454849 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:50.454849 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 331/652] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:50.455542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e78687-2f2c-4851-b674-f19ad1209640 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:50.480629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd08e1-3223-426d-b5bf-a710447db7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:50.481393 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd08e1-3223-426d-b5bf-a710447db7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:50.481751 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd08e1-3223-426d-b5bf-a710447db7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:50.482038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd08e1-3223-426d-b5bf-a710447db7f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:50.499659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:50.517159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:50.561986 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd08e1-3223-426d-b5bf-a710447db7f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:50.563456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd08e1-3223-426d-b5bf-a710447db7f6 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:50.579947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e78687-2f2c-4851-b674-f19ad1209640 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:50.618826 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528cc5a-9fdf-441a-ac65-f2decf02a02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:50.619743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528cc5a-9fdf-441a-ac65-f2decf02a02b None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:50.620239 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528cc5a-9fdf-441a-ac65-f2decf02a02b None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:50.620609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528cc5a-9fdf-441a-ac65-f2decf02a02b None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:50.621529 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528cc5a-9fdf-441a-ac65-f2decf02a02b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:50.621915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528cc5a-9fdf-441a-ac65-f2decf02a02b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:50.623955 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528cc5a-9fdf-441a-ac65-f2decf02a02b None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:50.637251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 332/653] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:50.638106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528cc5a-9fdf-441a-ac65-f2decf02a02b None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:50.685891 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 322/654] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:50.686763 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd08e1-3223-426d-b5bf-a710447db7f6 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:50.751729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:50.753071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528cc5a-9fdf-441a-ac65-f2decf02a02b None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:50.765497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:50.796066 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff3fcb-ca31-4065-87a5-905466dbcc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:50.797224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff3fcb-ca31-4065-87a5-905466dbcc67 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:50.798127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff3fcb-ca31-4065-87a5-905466dbcc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:50.798516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff3fcb-ca31-4065-87a5-905466dbcc67 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:50.799661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff3fcb-ca31-4065-87a5-905466dbcc67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:50.800072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff3fcb-ca31-4065-87a5-905466dbcc67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:50.802641 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff3fcb-ca31-4065-87a5-905466dbcc67 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:50.821061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd08e1-3223-426d-b5bf-a710447db7f6 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:50.821518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 333/655] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:50.822500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff3fcb-ca31-4065-87a5-905466dbcc67 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:50.865186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1c804-ba02-4b22-a544-284f135d2a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:50.866390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1c804-ba02-4b22-a544-284f135d2a91 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:50.866880 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1c804-ba02-4b22-a544-284f135d2a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:50.867349 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1c804-ba02-4b22-a544-284f135d2a91 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:50.875915 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1c804-ba02-4b22-a544-284f135d2a91 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:50.877751 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1c804-ba02-4b22-a544-284f135d2a91 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-b1b1c804-ba02-4b22-a544-284f135d2a91 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:50.898257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:50.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:50.900161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:50.900161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:50.900161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:50.900161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:50.900161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:50.900161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:50.900161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:50.900161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:50.900161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 15 10:02:50.900161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:50.900161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 323/656] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:50.900784 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1c804-ba02-4b22-a544-284f135d2a91 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:50.950683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff3fcb-ca31-4065-87a5-905466dbcc67 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:50.987064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33281b6-f382-48e7-9bae-ccb23636ddd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:50.987819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33281b6-f382-48e7-9bae-ccb23636ddd9 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:50.988105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33281b6-f382-48e7-9bae-ccb23636ddd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:50.988382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33281b6-f382-48e7-9bae-ccb23636ddd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:50.999868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33281b6-f382-48e7-9bae-ccb23636ddd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:51.001382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33281b6-f382-48e7-9bae-ccb23636ddd9 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:51.021383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-c33281b6-f382-48e7-9bae-ccb23636ddd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:51.021383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:51.021383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:51.021383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:51.021383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:51.021383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:51.021383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:51.021383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:51.021383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:51.021383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:51.021383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:51.021383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:51.021383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:51.021383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:51.021383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:51.022317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:51.023193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:51.023193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:51.023193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:51.023193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:51.023193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 334/657] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:51.023896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33281b6-f382-48e7-9bae-ccb23636ddd9 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:51.047455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1c804-ba02-4b22-a544-284f135d2a91 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:51.051303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:51.052440 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:51.095683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe3d68-4ca3-4562-8327-2769525c7cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:51.096877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe3d68-4ca3-4562-8327-2769525c7cec None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:51.097500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe3d68-4ca3-4562-8327-2769525c7cec None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:51.097926 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe3d68-4ca3-4562-8327-2769525c7cec None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:51.107928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe3d68-4ca3-4562-8327-2769525c7cec None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:51.112787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe3d68-4ca3-4562-8327-2769525c7cec None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-96fe3d68-4ca3-4562-8327-2769525c7cec None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:51.132036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:51.133171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:51.134258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:51.134258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:51.134258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:51.134258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:51.134258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:51.134258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:51.134258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:51.134258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:51.134258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:02:51.134258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:51.136257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 324/658] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:51.137364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe3d68-4ca3-4562-8327-2769525c7cec None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:51.167993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33281b6-f382-48e7-9bae-ccb23636ddd9 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:51.212811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4143a2f-46bb-4aad-ad7a-b9ada8eef4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:51.213745 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4143a2f-46bb-4aad-ad7a-b9ada8eef4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:51.214148 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4143a2f-46bb-4aad-ad7a-b9ada8eef4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:51.214522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4143a2f-46bb-4aad-ad7a-b9ada8eef4e5 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:51.215540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4143a2f-46bb-4aad-ad7a-b9ada8eef4e5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:51.215947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4143a2f-46bb-4aad-ad7a-b9ada8eef4e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:51.218404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4143a2f-46bb-4aad-ad7a-b9ada8eef4e5 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:51.240768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 335/659] 10.222.0.60 () {56 vars in 1139 bytes} [Wed Jun 15 10:02:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:51.241863 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4143a2f-46bb-4aad-ad7a-b9ada8eef4e5 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:51.281117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe3d68-4ca3-4562-8327-2769525c7cec None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:51.324227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79e587-f09a-4fc7-9a13-56146777d387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:51.325674 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79e587-f09a-4fc7-9a13-56146777d387 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:51.326067 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79e587-f09a-4fc7-9a13-56146777d387 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:51.326985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79e587-f09a-4fc7-9a13-56146777d387 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:51.327826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79e587-f09a-4fc7-9a13-56146777d387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:51.328513 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79e587-f09a-4fc7-9a13-56146777d387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:51.330289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79e587-f09a-4fc7-9a13-56146777d387 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:51.345220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:51.352271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 325/660] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:51.352271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79e587-f09a-4fc7-9a13-56146777d387 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:51.374569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4143a2f-46bb-4aad-ad7a-b9ada8eef4e5 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:51.389719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:51.422401 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5bcb1-3401-4ab3-828f-8201dbb22438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:51.424017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5bcb1-3401-4ab3-828f-8201dbb22438 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:51.424907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5bcb1-3401-4ab3-828f-8201dbb22438 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:51.425688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5bcb1-3401-4ab3-828f-8201dbb22438 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:51.426808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5bcb1-3401-4ab3-828f-8201dbb22438 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:51.427331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5bcb1-3401-4ab3-828f-8201dbb22438 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:51.429961 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5bcb1-3401-4ab3-828f-8201dbb22438 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:51.454706 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 336/661] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Jun 15 10:02:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:51.455898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5bcb1-3401-4ab3-828f-8201dbb22438 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:51.485838 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79e587-f09a-4fc7-9a13-56146777d387 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:51.523507 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf2160-caef-47fe-accd-849613c480a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:51.524261 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf2160-caef-47fe-accd-849613c480a9 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:51.524547 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf2160-caef-47fe-accd-849613c480a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:51.524821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf2160-caef-47fe-accd-849613c480a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:51.573740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5bcb1-3401-4ab3-828f-8201dbb22438 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:51.602236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf2160-caef-47fe-accd-849613c480a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:51.603687 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf2160-caef-47fe-accd-849613c480a9 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:51.613140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88464103-88b1-4c00-af9c-a29be238492d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:51.613944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88464103-88b1-4c00-af9c-a29be238492d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:51.614228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88464103-88b1-4c00-af9c-a29be238492d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:51.614506 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88464103-88b1-4c00-af9c-a29be238492d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:51.622442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88464103-88b1-4c00-af9c-a29be238492d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:51.624380 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88464103-88b1-4c00-af9c-a29be238492d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:51.626305 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:51.631965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:51.642055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-88464103-88b1-4c00-af9c-a29be238492d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:51.642055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:51.642055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:51.642055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:51.642055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:51.642055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:51.642055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:51.642055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:51.642055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:51.642055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:51.642055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:51.642055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:51.642055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:51.642055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:51.642055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:51.643224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:51.644268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:51.644268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:51.644268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:51.644268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:51.644268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:51.644268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:51.644268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:51.644268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:51.646355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 337/662] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:51.647437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88464103-88b1-4c00-af9c-a29be238492d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:51.736998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 326/663] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:51.736998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf2160-caef-47fe-accd-849613c480a9 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:51.781108 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88464103-88b1-4c00-af9c-a29be238492d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:51.819790 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54089a6-ab7a-4f7c-9b64-8578bd2ec024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:51.820455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54089a6-ab7a-4f7c-9b64-8578bd2ec024 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:51.820742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54089a6-ab7a-4f7c-9b64-8578bd2ec024 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:51.821058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54089a6-ab7a-4f7c-9b64-8578bd2ec024 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:51.863267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf2160-caef-47fe-accd-849613c480a9 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:51.889777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:51.901285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36dc5b-ed24-4e66-b6dc-a97b59f8f513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:51.901961 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36dc5b-ed24-4e66-b6dc-a97b59f8f513 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:51.902246 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36dc5b-ed24-4e66-b6dc-a97b59f8f513 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:51.902538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36dc5b-ed24-4e66-b6dc-a97b59f8f513 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:51.902730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:51.903325 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36dc5b-ed24-4e66-b6dc-a97b59f8f513 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:51.903637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36dc5b-ed24-4e66-b6dc-a97b59f8f513 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:51.905970 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36dc5b-ed24-4e66-b6dc-a97b59f8f513 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:51.907667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54089a6-ab7a-4f7c-9b64-8578bd2ec024 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:51.909171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54089a6-ab7a-4f7c-9b64-8578bd2ec024 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:51.925166 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 327/664] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:51.925166 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36dc5b-ed24-4e66-b6dc-a97b59f8f513 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:52.023177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 338/665] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:52.024449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b358f01-4ad3-4da9-9366-5847c9aa18ec None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:52.024742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b358f01-4ad3-4da9-9366-5847c9aa18ec None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:52.025181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b358f01-4ad3-4da9-9366-5847c9aa18ec None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:52.026430 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 339/666] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:52.027009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b358f01-4ad3-4da9-9366-5847c9aa18ec None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:52.060343 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36dc5b-ed24-4e66-b6dc-a97b59f8f513 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:52.097049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249518af-5475-49d2-808b-b749fb747822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:52.097859 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249518af-5475-49d2-808b-b749fb747822 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:52.098147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249518af-5475-49d2-808b-b749fb747822 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:52.098427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249518af-5475-49d2-808b-b749fb747822 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:52.109872 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249518af-5475-49d2-808b-b749fb747822 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:52.111420 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249518af-5475-49d2-808b-b749fb747822 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:52.129901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-249518af-5475-49d2-808b-b749fb747822 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:52.129901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:52.129901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:52.129901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:52.129901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:52.129901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:52.129901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:52.129901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:52.129901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:52.129901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:52.129901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:52.129901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:52.129901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:52.129901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:52.129901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:52.130865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:52.132084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:52.132084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:52.132084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:52.132084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:52.132084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 328/667] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:52.132497 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249518af-5475-49d2-808b-b749fb747822 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:52.145759 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:52.151577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b358f01-4ad3-4da9-9366-5847c9aa18ec None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:52.175331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:52.190268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59a6ab-6144-4a5a-b54d-865cf2609161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:52.190925 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59a6ab-6144-4a5a-b54d-865cf2609161 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:52.191210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59a6ab-6144-4a5a-b54d-865cf2609161 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:52.191500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59a6ab-6144-4a5a-b54d-865cf2609161 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:52.192328 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59a6ab-6144-4a5a-b54d-865cf2609161 None admin] RBAC: Adding query filter params (role.id=2c276d442f7f4dd59ae7fc3a03943716, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=7fbf33189aa94f4f973d558d615a23e6) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:52.192639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59a6ab-6144-4a5a-b54d-865cf2609161 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:52.194118 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59a6ab-6144-4a5a-b54d-865cf2609161 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:52.205008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 340/668] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:02:52 2022] GET /identity/v3/role_assignments?user.id=7fbf33189aa94f4f973d558d615a23e6&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=2c276d442f7f4dd59ae7fc3a03943716 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:52.205008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9008f7-9c7d-4a4a-98ea-ff835bd4d259 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:52.205008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9008f7-9c7d-4a4a-98ea-ff835bd4d259 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:52.205008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9008f7-9c7d-4a4a-98ea-ff835bd4d259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:52.230032 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-ae9008f7-9c7d-4a4a-98ea-ff835bd4d259 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:52.231869 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ae9008f7-9c7d-4a4a-98ea-ff835bd4d259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:52.259527 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249518af-5475-49d2-808b-b749fb747822 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:52.306265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de30d61-288c-43bb-99ed-7ab3cad324ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:52.307748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de30d61-288c-43bb-99ed-7ab3cad324ec None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:52.308046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de30d61-288c-43bb-99ed-7ab3cad324ec None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:52.308328 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de30d61-288c-43bb-99ed-7ab3cad324ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:52.327324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de30d61-288c-43bb-99ed-7ab3cad324ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:52.329118 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de30d61-288c-43bb-99ed-7ab3cad324ec None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:52.332075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 341/669] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:52.333234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9008f7-9c7d-4a4a-98ea-ff835bd4d259 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:52.365586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-8de30d61-288c-43bb-99ed-7ab3cad324ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:52.365586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:52.365586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:52.365586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:52.365586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:52.365586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:52.365586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:52.365586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:52.365586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:52.365586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:52.365586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:52.365586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:52.365586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:52.365586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:52.365586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:52.366700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:52.367606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:52.367606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:52.367606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:52.367606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:52.371728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 329/670] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:52.372731 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de30d61-288c-43bb-99ed-7ab3cad324ec None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:52.479123 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:52.480248 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:52.492104 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9008f7-9c7d-4a4a-98ea-ff835bd4d259 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:52.494819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de30d61-288c-43bb-99ed-7ab3cad324ec None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:52.538866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9698f8b-9e0b-413a-9abc-a9e3ef056602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:52.539794 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9698f8b-9e0b-413a-9abc-a9e3ef056602 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:52.540579 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9698f8b-9e0b-413a-9abc-a9e3ef056602 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:52.540579 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9698f8b-9e0b-413a-9abc-a9e3ef056602 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:52.541775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e372467-1872-473e-9a86-5b9c929f3598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:52.542472 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e372467-1872-473e-9a86-5b9c929f3598 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:52.542832 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e372467-1872-473e-9a86-5b9c929f3598 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:52.543068 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e372467-1872-473e-9a86-5b9c929f3598 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:52.543774 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e372467-1872-473e-9a86-5b9c929f3598 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:52.544105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e372467-1872-473e-9a86-5b9c929f3598 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:52.545836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e372467-1872-473e-9a86-5b9c929f3598 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:52.555047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9698f8b-9e0b-413a-9abc-a9e3ef056602 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:52.564131 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 342/671] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:52.564131 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e372467-1872-473e-9a86-5b9c929f3598 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-d9698f8b-9e0b-413a-9abc-a9e3ef056602 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:52.564386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:52.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:52.566366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:52.566366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:52.566366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:52.566366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:52.566366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:52.566366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:52.566366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:52.566366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:52.566366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:52.566366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:52.566366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:52.566366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:52.566366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 330/672] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:52.567040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76617-6fc0-4fdb-83df-5a413a4cb25f None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:52.567040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76617-6fc0-4fdb-83df-5a413a4cb25f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:52.567248 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76617-6fc0-4fdb-83df-5a413a4cb25f None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:52.572826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 331/673] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:52.573788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b76617-6fc0-4fdb-83df-5a413a4cb25f None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:52.681037 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e372467-1872-473e-9a86-5b9c929f3598 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:52.686460 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b76617-6fc0-4fdb-83df-5a413a4cb25f None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:52.722374 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a291e-758e-4ecb-9a01-7efe69af942f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:52.723047 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a291e-758e-4ecb-9a01-7efe69af942f None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:52.723335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a291e-758e-4ecb-9a01-7efe69af942f None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:52.723622 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a291e-758e-4ecb-9a01-7efe69af942f None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:52.724363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a291e-758e-4ecb-9a01-7efe69af942f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:52.724673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a291e-758e-4ecb-9a01-7efe69af942f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:52.726125 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a291e-758e-4ecb-9a01-7efe69af942f None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:52.729347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c9e09-23c8-4c97-8cb0-9040e49d5441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:52.730160 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c9e09-23c8-4c97-8cb0-9040e49d5441 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:52.730444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c9e09-23c8-4c97-8cb0-9040e49d5441 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:52.730720 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c9e09-23c8-4c97-8cb0-9040e49d5441 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:52.744244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c9e09-23c8-4c97-8cb0-9040e49d5441 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:52.744443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 343/674] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:52.744443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8194e5-0a4d-4a84-a392-7dfac4bea622 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:52.744633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9c9e09-23c8-4c97-8cb0-9040e49d5441 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:52.744718 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8194e5-0a4d-4a84-a392-7dfac4bea622 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:52.744718 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8194e5-0a4d-4a84-a392-7dfac4bea622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:52.761109 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-bd8194e5-0a4d-4a84-a392-7dfac4bea622 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:52.761522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-5d9c9e09-23c8-4c97-8cb0-9040e49d5441 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:52.761522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:52.761522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:52.761522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:52.761522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:52.761522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:52.761522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:52.761522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:52.761522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:52.761522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:52.761522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:52.761522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:52.761522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:52.761522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:52.761522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:52.762414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:52.763296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:52.763296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:52.763296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:52.763296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:52.763296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:52.763296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:52.763296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:52.763296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:52.763936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-bd8194e5-0a4d-4a84-a392-7dfac4bea622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:52.764062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 332/675] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:52.764062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c9e09-23c8-4c97-8cb0-9040e49d5441 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:52.774667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:52.779716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:52.845251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 344/676] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:52.845251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8194e5-0a4d-4a84-a392-7dfac4bea622 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:52.882056 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c9e09-23c8-4c97-8cb0-9040e49d5441 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:52.921075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06310a60-570f-426e-8308-582fa08d0946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:52.921751 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06310a60-570f-426e-8308-582fa08d0946 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:52.922034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06310a60-570f-426e-8308-582fa08d0946 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:52.922442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06310a60-570f-426e-8308-582fa08d0946 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:52.923220 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06310a60-570f-426e-8308-582fa08d0946 None admin] RBAC: Adding query filter params (name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:52.923620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06310a60-570f-426e-8308-582fa08d0946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:52.925028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06310a60-570f-426e-8308-582fa08d0946 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:52.934474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 333/677] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:52.935304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06310a60-570f-426e-8308-582fa08d0946 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:52.962747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8194e5-0a4d-4a84-a392-7dfac4bea622 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:53.000312 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b04b5-5162-4213-89f1-338c8c9a0375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:53.001153 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b04b5-5162-4213-89f1-338c8c9a0375 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:53.001432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b04b5-5162-4213-89f1-338c8c9a0375 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:53.001712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b04b5-5162-4213-89f1-338c8c9a0375 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:53.009200 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b04b5-5162-4213-89f1-338c8c9a0375 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:53.010690 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b04b5-5162-4213-89f1-338c8c9a0375 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:53.012381 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:53.025524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-eb8b04b5-5162-4213-89f1-338c8c9a0375 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:53.025524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:53.025524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:53.025524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:53.025524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:53.025524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:53.025524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:53.025524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:53.025524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:53.025524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:53.025524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:53.025524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:53.025524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:53.025524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:53.025524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:53.026949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:53.027859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:53.027859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:53.027859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:53.027859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:53.027859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:53.027859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:53.027859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:53.027859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:53.027859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 345/678] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:53.027859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b04b5-5162-4213-89f1-338c8c9a0375 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:53.034050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:53.054467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06310a60-570f-426e-8308-582fa08d0946 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:53.094295 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e84868-89bc-4d0b-9afe-a574e148f9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:53.094961 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e84868-89bc-4d0b-9afe-a574e148f9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:53.095242 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e84868-89bc-4d0b-9afe-a574e148f9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:53.095564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e84868-89bc-4d0b-9afe-a574e148f9d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:53.096310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e84868-89bc-4d0b-9afe-a574e148f9d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:53.096617 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e84868-89bc-4d0b-9afe-a574e148f9d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:53.098088 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e84868-89bc-4d0b-9afe-a574e148f9d5 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:53.116184 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 334/679] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:53.116184 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e84868-89bc-4d0b-9afe-a574e148f9d5 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:53.145714 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b04b5-5162-4213-89f1-338c8c9a0375 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:53.183670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcd0e8-5589-46e2-bc71-650fe94e1caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:53.184478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcd0e8-5589-46e2-bc71-650fe94e1caf None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:53.184765 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcd0e8-5589-46e2-bc71-650fe94e1caf None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:53.185072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcd0e8-5589-46e2-bc71-650fe94e1caf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:53.197228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcd0e8-5589-46e2-bc71-650fe94e1caf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-d8fcd0e8-5589-46e2-bc71-650fe94e1caf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:53.205980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:53.206910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:53.207733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:53.207733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:53.207733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:53.207733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:53.207733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:53.207733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:53.207733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:53.207733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:53.207733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:53.207733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:53.207733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:02:53.207733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:53.207733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 346/680] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:02:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:53.208415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcd0e8-5589-46e2-bc71-650fe94e1caf None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:53.236782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e84868-89bc-4d0b-9afe-a574e148f9d5 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:53.270415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:53.276377 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d97c29-cf6b-4d13-bfae-304f467e3afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:53.277211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d97c29-cf6b-4d13-bfae-304f467e3afd None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:53.283549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d97c29-cf6b-4d13-bfae-304f467e3afd None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:53.283549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d97c29-cf6b-4d13-bfae-304f467e3afd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:53.283798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:53.289337 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d97c29-cf6b-4d13-bfae-304f467e3afd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:53.290832 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d97c29-cf6b-4d13-bfae-304f467e3afd None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:53.311682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-f2d97c29-cf6b-4d13-bfae-304f467e3afd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:53.311682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:53.311682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:53.311682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:53.311682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:53.311682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:53.311682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:53.311682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:53.311682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:53.311682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:53.311682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:53.311682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:53.311682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:53.311682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:53.311682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:53.312644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:53.313565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:53.313565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:53.313565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:53.313565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:53.319341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 335/681] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:53.319341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d97c29-cf6b-4d13-bfae-304f467e3afd None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:53.321136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcd0e8-5589-46e2-bc71-650fe94e1caf None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:53.361212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e471e-b78b-4f03-bf83-bbed9cecfac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:53.361854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e471e-b78b-4f03-bf83-bbed9cecfac9 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:53.362139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e471e-b78b-4f03-bf83-bbed9cecfac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:53.362415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e471e-b78b-4f03-bf83-bbed9cecfac9 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:53.363261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e471e-b78b-4f03-bf83-bbed9cecfac9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:53.363572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e471e-b78b-4f03-bf83-bbed9cecfac9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:53.365033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e471e-b78b-4f03-bf83-bbed9cecfac9 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:53.383755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 347/682] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:53.383755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e471e-b78b-4f03-bf83-bbed9cecfac9 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:53.434457 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d97c29-cf6b-4d13-bfae-304f467e3afd None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:53.476570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988afc3-1d58-4bc5-81e7-98ff8a0887a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:53.477291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988afc3-1d58-4bc5-81e7-98ff8a0887a6 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:53.477577 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988afc3-1d58-4bc5-81e7-98ff8a0887a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:53.477854 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988afc3-1d58-4bc5-81e7-98ff8a0887a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:53.478734 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988afc3-1d58-4bc5-81e7-98ff8a0887a6 None admin] RBAC: Adding query filter params (role.id=2c276d442f7f4dd59ae7fc3a03943716, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=ae926ce83d464c768baddd90de31aa40) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:53.479047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988afc3-1d58-4bc5-81e7-98ff8a0887a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:53.481541 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988afc3-1d58-4bc5-81e7-98ff8a0887a6 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:53.493827 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e471e-b78b-4f03-bf83-bbed9cecfac9 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:53.501365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 336/683] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:02:53 2022] GET /identity/v3/role_assignments?user.id=ae926ce83d464c768baddd90de31aa40&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=2c276d442f7f4dd59ae7fc3a03943716 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:53.509037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988afc3-1d58-4bc5-81e7-98ff8a0887a6 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:53.548328 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:53.564588 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86395aa6-1d34-42cf-b65c-049a9787d0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:53.573923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86395aa6-1d34-42cf-b65c-049a9787d0ac None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:53.573923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86395aa6-1d34-42cf-b65c-049a9787d0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:53.573923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86395aa6-1d34-42cf-b65c-049a9787d0ac None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:53.573923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86395aa6-1d34-42cf-b65c-049a9787d0ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:53.573923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86395aa6-1d34-42cf-b65c-049a9787d0ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:53.573923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86395aa6-1d34-42cf-b65c-049a9787d0ac None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:53.587453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 348/684] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:53.589376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86395aa6-1d34-42cf-b65c-049a9787d0ac None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:53.592760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:53.689138 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988afc3-1d58-4bc5-81e7-98ff8a0887a6 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:53.744812 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110678ad-9bc5-4580-9f59-27c60c9fd80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:53.745326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110678ad-9bc5-4580-9f59-27c60c9fd80b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:53.745605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110678ad-9bc5-4580-9f59-27c60c9fd80b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:53.745879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110678ad-9bc5-4580-9f59-27c60c9fd80b None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:53.746604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110678ad-9bc5-4580-9f59-27c60c9fd80b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:53.746914 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110678ad-9bc5-4580-9f59-27c60c9fd80b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:53.749082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110678ad-9bc5-4580-9f59-27c60c9fd80b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:53.755590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86395aa6-1d34-42cf-b65c-049a9787d0ac None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:53.761724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 337/685] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:53.762627 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110678ad-9bc5-4580-9f59-27c60c9fd80b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:53.832423 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5346b-ee09-4af3-8307-126cfc3578a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:53.833135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5346b-ee09-4af3-8307-126cfc3578a8 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:53.833417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5346b-ee09-4af3-8307-126cfc3578a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:53.833692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5346b-ee09-4af3-8307-126cfc3578a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:53.834534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5346b-ee09-4af3-8307-126cfc3578a8 None admin] RBAC: Adding query filter params (role.id=2c276d442f7f4dd59ae7fc3a03943716, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=fa51162a69d44127bd34df2f4653dc15) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:53.834996 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5346b-ee09-4af3-8307-126cfc3578a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:53.836579 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5346b-ee09-4af3-8307-126cfc3578a8 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:53.845908 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 349/686] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:02:53 2022] GET /identity/v3/role_assignments?user.id=fa51162a69d44127bd34df2f4653dc15&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=2c276d442f7f4dd59ae7fc3a03943716 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:53.847020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5346b-ee09-4af3-8307-126cfc3578a8 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:53.915630 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110678ad-9bc5-4580-9f59-27c60c9fd80b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:53.926361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:53.965370 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:53.980267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51251e0f-6c98-4f4e-a19a-c1dc83f23028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:53.981124 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51251e0f-6c98-4f4e-a19a-c1dc83f23028 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:53.981404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51251e0f-6c98-4f4e-a19a-c1dc83f23028 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:53.981673 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51251e0f-6c98-4f4e-a19a-c1dc83f23028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:53.995102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51251e0f-6c98-4f4e-a19a-c1dc83f23028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:53.996545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51251e0f-6c98-4f4e-a19a-c1dc83f23028 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:54.017801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-51251e0f-6c98-4f4e-a19a-c1dc83f23028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:54.017801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:54.017801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:54.017801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:54.017801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:54.017801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:54.017801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:54.017801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:54.017801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:54.017801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:54.017801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:54.017801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:54.017801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:54.017801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:54.017801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:54.018756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:54.019618 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:54.019618 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:54.019618 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:54.019618 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:54.019618 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 338/687] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:54.027655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51251e0f-6c98-4f4e-a19a-c1dc83f23028 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:54.085091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5346b-ee09-4af3-8307-126cfc3578a8 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:54.130474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3dec60-e2d2-4cfb-a641-e88b77bf5807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:54.131344 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3dec60-e2d2-4cfb-a641-e88b77bf5807 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:54.131631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3dec60-e2d2-4cfb-a641-e88b77bf5807 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:54.132060 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3dec60-e2d2-4cfb-a641-e88b77bf5807 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:54.142560 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3dec60-e2d2-4cfb-a641-e88b77bf5807 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:54.144046 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3dec60-e2d2-4cfb-a641-e88b77bf5807 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-2c3dec60-e2d2-4cfb-a641-e88b77bf5807 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:54.160436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:54.162761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:54.165101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:54.165101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:54.165101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:54.165101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:54.165101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:54.165101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:54.165101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:54.165101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:54.165101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:02:54.165101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:54.167978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 350/688] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:02:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:54.174314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3dec60-e2d2-4cfb-a641-e88b77bf5807 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:54.187523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51251e0f-6c98-4f4e-a19a-c1dc83f23028 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:54.242413 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31683c-aee0-400d-8c9e-73fd01780be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:54.242994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31683c-aee0-400d-8c9e-73fd01780be6 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:54.243275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31683c-aee0-400d-8c9e-73fd01780be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:54.243549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a31683c-aee0-400d-8c9e-73fd01780be6 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:54.244281 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31683c-aee0-400d-8c9e-73fd01780be6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:54.244587 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31683c-aee0-400d-8c9e-73fd01780be6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:54.246003 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a31683c-aee0-400d-8c9e-73fd01780be6 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:54.261151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 339/689] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:54.270099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31683c-aee0-400d-8c9e-73fd01780be6 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:54.298024 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:54.301035 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3dec60-e2d2-4cfb-a641-e88b77bf5807 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:54.314519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:54.343664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1793f09-c1e7-46e6-8900-7feeedb3cccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:54.344746 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1793f09-c1e7-46e6-8900-7feeedb3cccc None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:54.345283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1793f09-c1e7-46e6-8900-7feeedb3cccc None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:54.345770 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1793f09-c1e7-46e6-8900-7feeedb3cccc None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:54.346880 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1793f09-c1e7-46e6-8900-7feeedb3cccc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:54.347423 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1793f09-c1e7-46e6-8900-7feeedb3cccc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:54.349258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1793f09-c1e7-46e6-8900-7feeedb3cccc None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:54.371890 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 351/690] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Jun 15 10:02:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:54.378267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1793f09-c1e7-46e6-8900-7feeedb3cccc None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:54.421183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a31683c-aee0-400d-8c9e-73fd01780be6 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:54.500478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e55b99-3e36-4feb-a35b-7142c5a71dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:54.502992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e55b99-3e36-4feb-a35b-7142c5a71dfc None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:54.506371 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e55b99-3e36-4feb-a35b-7142c5a71dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:54.506821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e55b99-3e36-4feb-a35b-7142c5a71dfc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:54.520731 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e55b99-3e36-4feb-a35b-7142c5a71dfc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:54.524133 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e55b99-3e36-4feb-a35b-7142c5a71dfc None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-a8e55b99-3e36-4feb-a35b-7142c5a71dfc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:02:54.548386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:54.549400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:54.550324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:54.550324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:54.550324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:02:54.550324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:02:54.550324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:02:54.550324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:02:54.550324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:02:54.550324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:02:54.550324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:02:54.550324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:54.553962 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 340/691] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jun 15 10:02:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:54.559661 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e55b99-3e36-4feb-a35b-7142c5a71dfc None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:54.619247 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1793f09-c1e7-46e6-8900-7feeedb3cccc None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:54.655462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:54.694537 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e55b99-3e36-4feb-a35b-7142c5a71dfc None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:54.710999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518b66e-79a0-4e3c-9277-4c5747b1e031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:54.711672 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518b66e-79a0-4e3c-9277-4c5747b1e031 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:54.711958 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518b66e-79a0-4e3c-9277-4c5747b1e031 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:54.712293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518b66e-79a0-4e3c-9277-4c5747b1e031 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:54.760019 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:54.760255 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860271c-de9a-4ff9-8ca4-5916680ccaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:54.760818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860271c-de9a-4ff9-8ca4-5916680ccaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:54.761153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860271c-de9a-4ff9-8ca4-5916680ccaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:54.761428 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860271c-de9a-4ff9-8ca4-5916680ccaf2 None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:54.762294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860271c-de9a-4ff9-8ca4-5916680ccaf2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:54.762607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860271c-de9a-4ff9-8ca4-5916680ccaf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:54.764615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860271c-de9a-4ff9-8ca4-5916680ccaf2 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:54.786704 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 341/692] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jun 15 10:02:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:54.788204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426d182-6981-472d-af77-41eaf7124a8a None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:54.788493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426d182-6981-472d-af77-41eaf7124a8a None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:54.788765 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426d182-6981-472d-af77-41eaf7124a8a None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:54.794262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 342/693] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:54.795727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426d182-6981-472d-af77-41eaf7124a8a None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:54.809969 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6518b66e-79a0-4e3c-9277-4c5747b1e031 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:54.812711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6518b66e-79a0-4e3c-9277-4c5747b1e031 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:54.911181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426d182-6981-472d-af77-41eaf7124a8a None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:54.936252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 352/694] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:54.938282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9245d0-2ae6-4752-b058-a061f18e497c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:54.938787 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9245d0-2ae6-4752-b058-a061f18e497c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:54.939441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9245d0-2ae6-4752-b058-a061f18e497c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:54.948157 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a2340-52ec-4b52-ad75-fa5865f435fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:54.948812 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a2340-52ec-4b52-ad75-fa5865f435fd None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:54.949125 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a2340-52ec-4b52-ad75-fa5865f435fd None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:54.949403 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a2340-52ec-4b52-ad75-fa5865f435fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:54.953862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:54.960693 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-ad9245d0-2ae6-4752-b058-a061f18e497c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:54.962327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ad9245d0-2ae6-4752-b058-a061f18e497c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:55.022468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a2340-52ec-4b52-ad75-fa5865f435fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:55.023857 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a2340-52ec-4b52-ad75-fa5865f435fd None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:55.032740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:55.041250 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 353/695] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:55.042903 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9245d0-2ae6-4752-b058-a061f18e497c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:55.156032 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9245d0-2ae6-4752-b058-a061f18e497c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:55.156906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 343/696] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:02:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:02:55.157798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a2340-52ec-4b52-ad75-fa5865f435fd None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:55.197853 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfedb2-b6d2-42f2-b0cc-b69b8f2af906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:55.199166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfedb2-b6d2-42f2-b0cc-b69b8f2af906 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:55.199797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:55.201121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfedb2-b6d2-42f2-b0cc-b69b8f2af906 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:55.201735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfedb2-b6d2-42f2-b0cc-b69b8f2af906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:55.214783 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfedb2-b6d2-42f2-b0cc-b69b8f2af906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:55.216857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfedb2-b6d2-42f2-b0cc-b69b8f2af906 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:55.236998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-04dfedb2-b6d2-42f2-b0cc-b69b8f2af906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:55.236998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:55.236998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:55.236998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:55.236998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:55.236998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:55.236998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:55.236998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:55.236998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:55.236998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:55.236998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:55.236998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:55.236998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:55.236998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:55.236998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:55.237943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:55.238844 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:55.238844 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:55.238844 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:55.238844 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:55.245904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 354/697] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:55.249021 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfedb2-b6d2-42f2-b0cc-b69b8f2af906 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:55.270668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a2340-52ec-4b52-ad75-fa5865f435fd None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:55.307292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9606b-f559-4f1a-9fae-35799795ba5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:55.307868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:55.308069 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9606b-f559-4f1a-9fae-35799795ba5f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:55.308367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9606b-f559-4f1a-9fae-35799795ba5f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:55.308671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9606b-f559-4f1a-9fae-35799795ba5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:55.320501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9606b-f559-4f1a-9fae-35799795ba5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:55.326785 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9606b-f559-4f1a-9fae-35799795ba5f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:55.341728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-faf9606b-f559-4f1a-9fae-35799795ba5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:55.341728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:55.341728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:55.341728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:55.341728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:55.341728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:55.341728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:55.341728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:55.341728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:55.341728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:55.341728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:55.341728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:55.341728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:55.341728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:55.341728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:55.343814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:55.344708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:55.344708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:55.344708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:55.344708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:55.348092 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 344/698] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:55.349198 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9606b-f559-4f1a-9fae-35799795ba5f None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:55.374935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfedb2-b6d2-42f2-b0cc-b69b8f2af906 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:55.416425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b895f8-0630-4a8a-9da8-521c57c8ecc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:55.417626 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b895f8-0630-4a8a-9da8-521c57c8ecc4 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:55.418219 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b895f8-0630-4a8a-9da8-521c57c8ecc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:55.418723 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b895f8-0630-4a8a-9da8-521c57c8ecc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:55.431058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b895f8-0630-4a8a-9da8-521c57c8ecc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:55.433048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b895f8-0630-4a8a-9da8-521c57c8ecc4 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:55.453457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-98b895f8-0630-4a8a-9da8-521c57c8ecc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:55.453457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:55.453457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:55.453457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:55.453457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:55.453457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:55.453457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:55.453457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:55.453457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:55.453457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:55.453457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:55.453457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:55.453457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:55.453457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:55.453457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:55.454407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:55.455286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:55.455286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:55.455286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:55.455286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:55.457602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 355/699] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:55.458663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b895f8-0630-4a8a-9da8-521c57c8ecc4 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:55.459283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9606b-f559-4f1a-9fae-35799795ba5f None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:55.462161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:55.496222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4c89a-70c5-4d58-b3ee-bd475bd8f048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:55.497038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4c89a-70c5-4d58-b3ee-bd475bd8f048 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:55.497397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4c89a-70c5-4d58-b3ee-bd475bd8f048 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:55.497743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4c89a-70c5-4d58-b3ee-bd475bd8f048 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:55.499142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4c89a-70c5-4d58-b3ee-bd475bd8f048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:55.499536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4c89a-70c5-4d58-b3ee-bd475bd8f048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:55.501440 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4c89a-70c5-4d58-b3ee-bd475bd8f048 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:55.518738 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 345/700] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:55.518738 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4c89a-70c5-4d58-b3ee-bd475bd8f048 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:55.585915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:55.589931 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b895f8-0630-4a8a-9da8-521c57c8ecc4 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:55.627963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4c89a-70c5-4d58-b3ee-bd475bd8f048 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:55.629968 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e104ae0-3e22-4ebc-8a01-047c031bc6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:55.631185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e104ae0-3e22-4ebc-8a01-047c031bc6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:55.631644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e104ae0-3e22-4ebc-8a01-047c031bc6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:55.632071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e104ae0-3e22-4ebc-8a01-047c031bc6e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:55.632977 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e104ae0-3e22-4ebc-8a01-047c031bc6e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:55.633435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e104ae0-3e22-4ebc-8a01-047c031bc6e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:55.634969 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e104ae0-3e22-4ebc-8a01-047c031bc6e9 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:55.648725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 356/701] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:55.650281 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e104ae0-3e22-4ebc-8a01-047c031bc6e9 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:55.670408 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9661b70-d2a7-48c1-a7c5-54f6859da9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:55.671201 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9661b70-d2a7-48c1-a7c5-54f6859da9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:55.671628 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9661b70-d2a7-48c1-a7c5-54f6859da9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:55.671911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9661b70-d2a7-48c1-a7c5-54f6859da9d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:55.672741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9661b70-d2a7-48c1-a7c5-54f6859da9d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:55.673099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9661b70-d2a7-48c1-a7c5-54f6859da9d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:55.674610 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9661b70-d2a7-48c1-a7c5-54f6859da9d5 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:55.688538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 346/702] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:55.689280 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9661b70-d2a7-48c1-a7c5-54f6859da9d5 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:55.700413 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:55.768778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e104ae0-3e22-4ebc-8a01-047c031bc6e9 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:55.795407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9661b70-d2a7-48c1-a7c5-54f6859da9d5 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:55.809284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9a93b-6c96-4797-90b2-d842a985f5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:55.811051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9a93b-6c96-4797-90b2-d842a985f5fa None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:55.811722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9a93b-6c96-4797-90b2-d842a985f5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:55.812229 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9a93b-6c96-4797-90b2-d842a985f5fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:55.821339 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9a93b-6c96-4797-90b2-d842a985f5fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:55.823182 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9a93b-6c96-4797-90b2-d842a985f5fa None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:55.838101 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3bf99-62e4-497b-a012-8149bdded16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:55.838758 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3bf99-62e4-497b-a012-8149bdded16b None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:55.839041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3bf99-62e4-497b-a012-8149bdded16b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:55.839318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3bf99-62e4-497b-a012-8149bdded16b None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:55.840152 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3bf99-62e4-497b-a012-8149bdded16b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:55.842250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f3bf99-62e4-497b-a012-8149bdded16b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:55.844376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-68a9a93b-6c96-4797-90b2-d842a985f5fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:55.844376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:55.844376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:55.844376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:55.844376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:55.844376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:55.844376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:55.844376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:55.844376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:55.844376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:55.844376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:55.844376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:55.844376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:55.844376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:55.844376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:55.845279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:55.846172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:55.846172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:55.846172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:55.846172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:55.846172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:55.846172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:55.846172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:55.846172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:55.853856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 357/703] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:55.854825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9a93b-6c96-4797-90b2-d842a985f5fa None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:55.879125 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:55.941432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 347/704] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:02:55 2022] POST /identity/v3/users => generated 254 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:02:55.942441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3bf99-62e4-497b-a012-8149bdded16b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:55.963385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9a93b-6c96-4797-90b2-d842a985f5fa None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:56.015232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609286a3-0813-4b7b-b3df-e4626c8f1f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:56.016459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609286a3-0813-4b7b-b3df-e4626c8f1f75 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:56.017049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609286a3-0813-4b7b-b3df-e4626c8f1f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:56.017559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609286a3-0813-4b7b-b3df-e4626c8f1f75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:56.029166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609286a3-0813-4b7b-b3df-e4626c8f1f75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:56.031750 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609286a3-0813-4b7b-b3df-e4626c8f1f75 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:56.048939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:56.055802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-609286a3-0813-4b7b-b3df-e4626c8f1f75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:56.055802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:56.055802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:56.055802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:56.055802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:56.055802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:56.055802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:56.055802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:56.055802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:56.055802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:56.055802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:56.055802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:56.055802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:02:56.055802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:02:56.055802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:56.056822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:56.057798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:02:56.057798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:02:56.057798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:02:56.057798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:02:56.057798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:02:56.057798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:02:56.057798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:02:56.057798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:02:56.058989 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 358/705] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:56.060059 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad1807-fa68-4106-aad5-7c13659efe9e None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:56.062597 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad1807-fa68-4106-aad5-7c13659efe9e None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:56.064389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad1807-fa68-4106-aad5-7c13659efe9e None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:56.071513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 359/706] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:56 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:56.072479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad1807-fa68-4106-aad5-7c13659efe9e None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:56.106073 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3bf99-62e4-497b-a012-8149bdded16b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:56.148583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18ed37-bdd1-4221-afe0-e6f4e5cf88b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:56.149635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18ed37-bdd1-4221-afe0-e6f4e5cf88b2 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:56.149926 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18ed37-bdd1-4221-afe0-e6f4e5cf88b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:56.150206 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18ed37-bdd1-4221-afe0-e6f4e5cf88b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:56.151026 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18ed37-bdd1-4221-afe0-e6f4e5cf88b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:56.151337 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18ed37-bdd1-4221-afe0-e6f4e5cf88b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:56.156067 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18ed37-bdd1-4221-afe0-e6f4e5cf88b2 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:56.172534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 348/707] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:56.175016 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400b6bd-79fb-4c61-a83f-58cebab60027 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:56.175777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400b6bd-79fb-4c61-a83f-58cebab60027 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:56.176281 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400b6bd-79fb-4c61-a83f-58cebab60027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:56.195237 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-c400b6bd-79fb-4c61-a83f-58cebab60027 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:56.196786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c400b6bd-79fb-4c61-a83f-58cebab60027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:56.198847 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:56.227164 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad1807-fa68-4106-aad5-7c13659efe9e None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:56.279617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1555bd-0622-4b9d-a753-209acf5f5cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:56.280725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1555bd-0622-4b9d-a753-209acf5f5cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:56.281228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1555bd-0622-4b9d-a753-209acf5f5cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:56.281727 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1555bd-0622-4b9d-a753-209acf5f5cf2 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:56.282784 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1555bd-0622-4b9d-a753-209acf5f5cf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:56.283242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1555bd-0622-4b9d-a753-209acf5f5cf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:56.284330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 349/708] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:56.285178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1555bd-0622-4b9d-a753-209acf5f5cf2 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:56.285410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400b6bd-79fb-4c61-a83f-58cebab60027 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:56.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 360/709] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:02:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:56.306565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1555bd-0622-4b9d-a753-209acf5f5cf2 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:56.353984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:56.404888 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400b6bd-79fb-4c61-a83f-58cebab60027 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:56.425800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1555bd-0622-4b9d-a753-209acf5f5cf2 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:56.443890 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c48cb-29a4-484b-8901-253f23e4ae10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:56.444903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c48cb-29a4-484b-8901-253f23e4ae10 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:56.445443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c48cb-29a4-484b-8901-253f23e4ae10 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:56.445906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c48cb-29a4-484b-8901-253f23e4ae10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:56.447030 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c48cb-29a4-484b-8901-253f23e4ae10 None admin] RBAC: Adding query filter params (role.id=2c276d442f7f4dd59ae7fc3a03943716, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=589e66c6228a44adbc9e418794d7d91b) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:56.447574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c48cb-29a4-484b-8901-253f23e4ae10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:56.449347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119c48cb-29a4-484b-8901-253f23e4ae10 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:56.463028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 350/710] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:02:56 2022] GET /identity/v3/role_assignments?user.id=589e66c6228a44adbc9e418794d7d91b&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=2c276d442f7f4dd59ae7fc3a03943716 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:56.464200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c48cb-29a4-484b-8901-253f23e4ae10 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:56.471437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e2aff-c57e-4645-90d3-2bc8df222058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:56.472476 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e2aff-c57e-4645-90d3-2bc8df222058 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:56.472763 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e2aff-c57e-4645-90d3-2bc8df222058 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:56.473061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e2aff-c57e-4645-90d3-2bc8df222058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:56.473965 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e2aff-c57e-4645-90d3-2bc8df222058 None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=26f23f8595fc43ad94aee7d0192164a1) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:56.476246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e2aff-c57e-4645-90d3-2bc8df222058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:56.480258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e2aff-c57e-4645-90d3-2bc8df222058 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:56.492940 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 361/711] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:02:56 2022] GET /identity/v3/role_assignments?user.id=26f23f8595fc43ad94aee7d0192164a1&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:56.576518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6cf31-f872-492f-b79c-ec8ae57ccfb2 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:56.576867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6cf31-f872-492f-b79c-ec8ae57ccfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:56.577152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f6cf31-f872-492f-b79c-ec8ae57ccfb2 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:56.578634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 362/712] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:02:56.586511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1a5b2-77c6-418d-91ab-5a2bd1a4b364 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:56.586822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1a5b2-77c6-418d-91ab-5a2bd1a4b364 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:56.587080 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1a5b2-77c6-418d-91ab-5a2bd1a4b364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:56.594634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:56.612046 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-fae1a5b2-77c6-418d-91ab-5a2bd1a4b364 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:56.613752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-fae1a5b2-77c6-418d-91ab-5a2bd1a4b364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:56.688893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c48cb-29a4-484b-8901-253f23e4ae10 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:56.719070 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:56.736292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735be67-e1d7-420f-8881-4ef7c3bd45e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:56.736569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 363/713] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:56.737158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735be67-e1d7-420f-8881-4ef7c3bd45e8 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:56.737445 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735be67-e1d7-420f-8881-4ef7c3bd45e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:56.737992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5735be67-e1d7-420f-8881-4ef7c3bd45e8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:56.738597 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5735be67-e1d7-420f-8881-4ef7c3bd45e8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:56.740450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5735be67-e1d7-420f-8881-4ef7c3bd45e8 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-5735be67-e1d7-420f-8881-4ef7c3bd45e8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:56.757136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:56.758262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:56.759188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:56.759188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:56.759188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:56.759188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:56.759188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:56.759188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:56.759188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:56.759188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:56.759188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:56.759188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:56.759188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:02:56.759188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:02:56.759188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:02:56.759188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:56.759188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 351/714] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:02:56 2022] GET /identity/v3/services/network => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:56.765240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1a5b2-77c6-418d-91ab-5a2bd1a4b364 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:56.766275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735be67-e1d7-420f-8881-4ef7c3bd45e8 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:56.878874 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5735be67-e1d7-420f-8881-4ef7c3bd45e8 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:56.886269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1a5b2-77c6-418d-91ab-5a2bd1a4b364 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:56.918355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc5415-c8be-4c19-b767-809653dc0c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:56.919504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc5415-c8be-4c19-b767-809653dc0c72 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:56.920539 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc5415-c8be-4c19-b767-809653dc0c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:56.920890 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc5415-c8be-4c19-b767-809653dc0c72 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:56.921661 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc5415-c8be-4c19-b767-809653dc0c72 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:56.923127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc5415-c8be-4c19-b767-809653dc0c72 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:56.934864 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33653cb3-250d-45a8-9457-aebeab0493e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:56.936411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33653cb3-250d-45a8-9457-aebeab0493e4 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:56.936927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33653cb3-250d-45a8-9457-aebeab0493e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:56.937438 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33653cb3-250d-45a8-9457-aebeab0493e4 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:56.938331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33653cb3-250d-45a8-9457-aebeab0493e4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:56.938788 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33653cb3-250d-45a8-9457-aebeab0493e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-b8bc5415-c8be-4c19-b767-809653dc0c72 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:56.939111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:56.940194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:56.941244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:56.941244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:56.941244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:56.941244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:56.941244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:56.941244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:56.941244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:56.941244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:56.941244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:56.941244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:56.941244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:56.941244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:02:56.941244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:02:56.941244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:02:56.941244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:56.943430 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 352/715] 10.222.0.60 () {56 vars in 1129 bytes} [Wed Jun 15 10:02:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:56.943716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33653cb3-250d-45a8-9457-aebeab0493e4 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:56.949193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 364/716] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jun 15 10:02:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:56.949375 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:56.950572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc5415-c8be-4c19-b767-809653dc0c72 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:56.954644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33653cb3-250d-45a8-9457-aebeab0493e4 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:57.061930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:57.109318 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33653cb3-250d-45a8-9457-aebeab0493e4 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:57.160828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3536c-42df-4722-8951-4599e583ab34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:57.162002 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3536c-42df-4722-8951-4599e583ab34 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:57.162456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3536c-42df-4722-8951-4599e583ab34 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:57.162894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3536c-42df-4722-8951-4599e583ab34 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:57.163781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3536c-42df-4722-8951-4599e583ab34 None admin] RBAC: Adding query filter params (type=network) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:57.164255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3536c-42df-4722-8951-4599e583ab34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:57.165954 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3536c-42df-4722-8951-4599e583ab34 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:57.172055 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc5415-c8be-4c19-b767-809653dc0c72 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:57.173051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 365/717] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jun 15 10:02:56 2022] GET /identity/v3/services?type=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:02:57.229705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fca3d3-5c3d-4793-940e-3c481b04c894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:57.230392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fca3d3-5c3d-4793-940e-3c481b04c894 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:57.230676 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fca3d3-5c3d-4793-940e-3c481b04c894 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:57.230955 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fca3d3-5c3d-4793-940e-3c481b04c894 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:57.231746 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fca3d3-5c3d-4793-940e-3c481b04c894 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:57.232061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fca3d3-5c3d-4793-940e-3c481b04c894 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:57.233580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fca3d3-5c3d-4793-940e-3c481b04c894 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:57.244069 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 353/718] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Jun 15 10:02:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:57.245552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3536c-42df-4722-8951-4599e583ab34 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:57.353326 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:57.401772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3536c-42df-4722-8951-4599e583ab34 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:57.451472 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91fcbf-6df6-42d5-98d2-4a4883cf2640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:57.452549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91fcbf-6df6-42d5-98d2-4a4883cf2640 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:57.453177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91fcbf-6df6-42d5-98d2-4a4883cf2640 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:57.453658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91fcbf-6df6-42d5-98d2-4a4883cf2640 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:57.454576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91fcbf-6df6-42d5-98d2-4a4883cf2640 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:57.455102 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91fcbf-6df6-42d5-98d2-4a4883cf2640 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:57.456692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91fcbf-6df6-42d5-98d2-4a4883cf2640 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:57.464457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 366/719] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Jun 15 10:02:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:58.615837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe2f82-f782-443c-a263-4f3a4225d137 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:58.616359 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe2f82-f782-443c-a263-4f3a4225d137 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:58.616484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe2f82-f782-443c-a263-4f3a4225d137 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:58.618073 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 354/720] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:58.624306 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304db92-e300-4bf6-8e3c-fa4b9f558ac5 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:58.624810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304db92-e300-4bf6-8e3c-fa4b9f558ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:58.624902 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304db92-e300-4bf6-8e3c-fa4b9f558ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:58.650738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-d304db92-e300-4bf6-8e3c-fa4b9f558ac5 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:58.651947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d304db92-e300-4bf6-8e3c-fa4b9f558ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:58.725192 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 367/721] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:02:58.749573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe2f82-f782-443c-a263-4f3a4225d137 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:58.903950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe2f82-f782-443c-a263-4f3a4225d137 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:58.915543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:58.941072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028affd6-c133-4246-bb26-6e734d78ca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:58.942137 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028affd6-c133-4246-bb26-6e734d78ca9b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:58.942578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028affd6-c133-4246-bb26-6e734d78ca9b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:58.943006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028affd6-c133-4246-bb26-6e734d78ca9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:58.965303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028affd6-c133-4246-bb26-6e734d78ca9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-028affd6-c133-4246-bb26-6e734d78ca9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:58.973412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:58.974354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:58.975193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:58.975193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:58.975193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:58.975193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:58.975193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:58.975193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:58.975193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:58.975193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:58.975193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:58.975193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:58.975193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:02:58.975193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:58.975193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 355/722] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:02:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:58.980802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304db92-e300-4bf6-8e3c-fa4b9f558ac5 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:59.062345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:59.122710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304db92-e300-4bf6-8e3c-fa4b9f558ac5 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:59.167807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184fcce-e4c0-46fe-8a94-071650bdebbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:59.168430 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184fcce-e4c0-46fe-8a94-071650bdebbf None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:59.168719 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184fcce-e4c0-46fe-8a94-071650bdebbf None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:59.169036 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184fcce-e4c0-46fe-8a94-071650bdebbf None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:59.169872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184fcce-e4c0-46fe-8a94-071650bdebbf None admin] RBAC: Adding query filter params (name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:59.170323 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184fcce-e4c0-46fe-8a94-071650bdebbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:59.172046 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184fcce-e4c0-46fe-8a94-071650bdebbf None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:59.183241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 368/723] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:02:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:59.190701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028affd6-c133-4246-bb26-6e734d78ca9b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:59.213378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ee825-6bad-464a-b682-fd71db27b1e5 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:59.213697 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ee825-6bad-464a-b682-fd71db27b1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:59.213939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ee825-6bad-464a-b682-fd71db27b1e5 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:59.215684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 369/724] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:02:59.222216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9411e-a1c7-40cf-b986-35970d6e1ead None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:59.222557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9411e-a1c7-40cf-b986-35970d6e1ead None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:59.222785 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9411e-a1c7-40cf-b986-35970d6e1ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:59.250080 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-71c9411e-a1c7-40cf-b986-35970d6e1ead None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:59.251639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-71c9411e-a1c7-40cf-b986-35970d6e1ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:59.327817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028affd6-c133-4246-bb26-6e734d78ca9b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:59.336911 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 370/725] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 15 10:02:59.338367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf75916-9acf-4926-9fd2-7fbb33a1cd6c None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:59.338697 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf75916-9acf-4926-9fd2-7fbb33a1cd6c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:59.339330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf75916-9acf-4926-9fd2-7fbb33a1cd6c None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:59.341124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 371/726] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:02:59.354935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a8dd6-8778-4e7b-922d-f77b06f3c969 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:59.355155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a8dd6-8778-4e7b-922d-f77b06f3c969 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:59.355374 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a8dd6-8778-4e7b-922d-f77b06f3c969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:59.378976 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-3c2a8dd6-8778-4e7b-922d-f77b06f3c969 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:02:59.380526 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-3c2a8dd6-8778-4e7b-922d-f77b06f3c969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:02:59.383248 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab84c3-944a-4956-bdb5-ff5938d56674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:59.384574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab84c3-944a-4956-bdb5-ff5938d56674 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:59.384574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab84c3-944a-4956-bdb5-ff5938d56674 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:59.384919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab84c3-944a-4956-bdb5-ff5938d56674 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:59.397506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab84c3-944a-4956-bdb5-ff5938d56674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:59.399226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab84c3-944a-4956-bdb5-ff5938d56674 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:59.419649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-fcab84c3-944a-4956-bdb5-ff5938d56674 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:59.419649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:59.419649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:59.419649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:59.419649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:59.419649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:59.419649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:59.419649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:59.419649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:59.419649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:59.419649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:59.419649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:59.419649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:02:59.419649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:02:59.419649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:59.420554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:59.421484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:02:59.421484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:02:59.421484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:02:59.421484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:59.426131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 356/727] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:59.427129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab84c3-944a-4956-bdb5-ff5938d56674 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:59.433171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:59.456752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:59.460592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 372/728] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:02:59.465754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a8dd6-8778-4e7b-922d-f77b06f3c969 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:59.567778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab84c3-944a-4956-bdb5-ff5938d56674 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:59.604910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37ac07-4bd0-4fbf-88cd-dbf3eda02ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:59.606047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37ac07-4bd0-4fbf-88cd-dbf3eda02ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:59.606528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37ac07-4bd0-4fbf-88cd-dbf3eda02ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:59.607010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37ac07-4bd0-4fbf-88cd-dbf3eda02ae9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:59.619280 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37ac07-4bd0-4fbf-88cd-dbf3eda02ae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:59.622662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a8dd6-8778-4e7b-922d-f77b06f3c969 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-cb37ac07-4bd0-4fbf-88cd-dbf3eda02ae9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:59.628419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:59.629380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:59.630236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:59.630236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:59.630236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:59.630236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:59.630236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:59.630236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:59.630236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:59.630236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:59.630236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:59.630236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:59.630236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:02:59.630236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:59.633085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 357/729] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:02:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:59.638040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37ac07-4bd0-4fbf-88cd-dbf3eda02ae9 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:59.661875 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7012a-d416-4b39-b048-2428a0f87fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:59.663317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7012a-d416-4b39-b048-2428a0f87fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:59.663875 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7012a-d416-4b39-b048-2428a0f87fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:59.664450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7012a-d416-4b39-b048-2428a0f87fc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:59.665512 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7012a-d416-4b39-b048-2428a0f87fc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:59.666023 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7012a-d416-4b39-b048-2428a0f87fc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:59.667756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7012a-d416-4b39-b048-2428a0f87fc8 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:59.685490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 373/730] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:59.686968 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7012a-d416-4b39-b048-2428a0f87fc8 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:59.736902 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:59.750282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:02:59.776015 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37ac07-4bd0-4fbf-88cd-dbf3eda02ae9 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:59.819061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fe38f-2d9c-410c-9aa1-f354a9281f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:59.820222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fe38f-2d9c-410c-9aa1-f354a9281f9e None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:59.820331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7012a-d416-4b39-b048-2428a0f87fc8 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:02:59.820837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fe38f-2d9c-410c-9aa1-f354a9281f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:59.821348 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52fe38f-2d9c-410c-9aa1-f354a9281f9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:59.837667 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52fe38f-2d9c-410c-9aa1-f354a9281f9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-b52fe38f-2d9c-410c-9aa1-f354a9281f9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:02:59.849481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:02:59.850501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:02:59.851341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:02:59.851341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:02:59.851341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:02:59.851341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:02:59.851341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:02:59.851341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:02:59.851341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:02:59.851341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:02:59.851341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:02:59.851341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:02:59.851341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:02:59.851341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:02:59.854580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 358/731] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:02:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:02:59.855652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fe38f-2d9c-410c-9aa1-f354a9281f9e None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:59.866891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea90c1-07a0-4eab-af41-1c76405e8684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:02:59.867716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea90c1-07a0-4eab-af41-1c76405e8684 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:02:59.868122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea90c1-07a0-4eab-af41-1c76405e8684 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:02:59.868486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea90c1-07a0-4eab-af41-1c76405e8684 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:02:59.869572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea90c1-07a0-4eab-af41-1c76405e8684 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:02:59.869987 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea90c1-07a0-4eab-af41-1c76405e8684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:02:59.871652 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea90c1-07a0-4eab-af41-1c76405e8684 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:02:59.886645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 374/732] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:02:59.886645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea90c1-07a0-4eab-af41-1c76405e8684 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:02:59.976923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52fe38f-2d9c-410c-9aa1-f354a9281f9e None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:00.001324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:00.005707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea90c1-07a0-4eab-af41-1c76405e8684 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:00.019914 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040d85b-3f82-46f2-aa02-390ece091f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:00.021035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040d85b-3f82-46f2-aa02-390ece091f31 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:00.021500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040d85b-3f82-46f2-aa02-390ece091f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:00.021925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040d85b-3f82-46f2-aa02-390ece091f31 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:00.031287 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040d85b-3f82-46f2-aa02-390ece091f31 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:00.033150 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040d85b-3f82-46f2-aa02-390ece091f31 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:00.046976 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a79092-c67d-4149-9dad-f556abbd78f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:00.047731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a79092-c67d-4149-9dad-f556abbd78f0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:00.048020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a79092-c67d-4149-9dad-f556abbd78f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:00.048301 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a79092-c67d-4149-9dad-f556abbd78f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:00.049183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a79092-c67d-4149-9dad-f556abbd78f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:00.049498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a79092-c67d-4149-9dad-f556abbd78f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:00.051202 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a79092-c67d-4149-9dad-f556abbd78f0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-a040d85b-3f82-46f2-aa02-390ece091f31 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:03:00.052062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:00.052990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:00.053923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:00.053923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:00.053923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:03:00.053923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:03:00.053923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:03:00.053923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:03:00.053923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:03:00.053923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:03:00.053923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:03:00.053923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:00.055127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 359/733] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jun 15 10:02:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:00.055523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040d85b-3f82-46f2-aa02-390ece091f31 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:00.061072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:00.064283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 375/734] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:02:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:00.065814 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4b9ef-4cf1-43e1-ae8a-4db26b249f53 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:00.066796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4b9ef-4cf1-43e1-ae8a-4db26b249f53 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:00.067237 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4b9ef-4cf1-43e1-ae8a-4db26b249f53 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:00.069338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 376/735] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:00.070558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4b9ef-4cf1-43e1-ae8a-4db26b249f53 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:00.178737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040d85b-3f82-46f2-aa02-390ece091f31 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:00.186096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4b9ef-4cf1-43e1-ae8a-4db26b249f53 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:00.216977 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41b372-772d-436b-aa00-0aaacde69988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:00.217980 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41b372-772d-436b-aa00-0aaacde69988 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:00.218267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41b372-772d-436b-aa00-0aaacde69988 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:00.218705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41b372-772d-436b-aa00-0aaacde69988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:00.224250 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a1bad-53ec-42bd-8891-9244bb89b6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:00.224947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a1bad-53ec-42bd-8891-9244bb89b6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:00.225324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a1bad-53ec-42bd-8891-9244bb89b6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:00.225608 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a1bad-53ec-42bd-8891-9244bb89b6f1 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:00.227223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a1bad-53ec-42bd-8891-9244bb89b6f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:00.227498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a1bad-53ec-42bd-8891-9244bb89b6f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:00.229555 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a1bad-53ec-42bd-8891-9244bb89b6f1 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:00.231431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41b372-772d-436b-aa00-0aaacde69988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:00.233295 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41b372-772d-436b-aa00-0aaacde69988 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:00.250498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 377/736] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Jun 15 10:03:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:00.251661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a1bad-53ec-42bd-8891-9244bb89b6f1 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:00.254340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-1e41b372-772d-436b-aa00-0aaacde69988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:00.254340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:00.254340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:00.254340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:00.254340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:00.254340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:00.254340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:00.254340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:00.254340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:00.254340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:00.254340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:00.254340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:00.254340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:00.254340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:00.254340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:00.255244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:00.256141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:00.256141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:00.256141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:00.256141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:00.256141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 360/737] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:00.257616 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba931762-084d-4988-8b6c-9e05a7817665 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:00.257899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba931762-084d-4988-8b6c-9e05a7817665 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:00.258173 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba931762-084d-4988-8b6c-9e05a7817665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:00.276088 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-ba931762-084d-4988-8b6c-9e05a7817665 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:00.277410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ba931762-084d-4988-8b6c-9e05a7817665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:00.280884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:00.334268 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:00.353819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 361/738] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:03:00.354671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba931762-084d-4988-8b6c-9e05a7817665 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:00.364401 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a1bad-53ec-42bd-8891-9244bb89b6f1 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:00.401031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d3613-5a38-47b6-bba6-60609f2a767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:00.401835 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d3613-5a38-47b6-bba6-60609f2a767e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:00.402118 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d3613-5a38-47b6-bba6-60609f2a767e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:00.402393 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d3613-5a38-47b6-bba6-60609f2a767e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:00.413677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d3613-5a38-47b6-bba6-60609f2a767e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:00.415144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d3613-5a38-47b6-bba6-60609f2a767e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:00.436345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-c14d3613-5a38-47b6-bba6-60609f2a767e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:00.436345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:00.436345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:00.436345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:00.436345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:00.436345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:00.436345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:00.436345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:00.436345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:00.436345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:00.436345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:00.436345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:00.436345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:00.436345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:00.436345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:00.437377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:00.438269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:00.438269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:00.438269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:00.438269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:00.438269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 378/739] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:00.438553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d3613-5a38-47b6-bba6-60609f2a767e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:00.468456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba931762-084d-4988-8b6c-9e05a7817665 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:00.504273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f27d8e-da01-43af-8cc5-a1b28dc306d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:00.504842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f27d8e-da01-43af-8cc5-a1b28dc306d5 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:00.505158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f27d8e-da01-43af-8cc5-a1b28dc306d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:00.505438 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f27d8e-da01-43af-8cc5-a1b28dc306d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:00.511184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:00.542549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d3613-5a38-47b6-bba6-60609f2a767e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:00.565839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:00.578072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f044c73d-7861-498a-9a53-5b02d0714a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:00.578749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044c73d-7861-498a-9a53-5b02d0714a51 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:00.579036 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044c73d-7861-498a-9a53-5b02d0714a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:00.579317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f044c73d-7861-498a-9a53-5b02d0714a51 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:00.580059 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f044c73d-7861-498a-9a53-5b02d0714a51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:00.580370 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f044c73d-7861-498a-9a53-5b02d0714a51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:00.581817 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f044c73d-7861-498a-9a53-5b02d0714a51 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:00.582149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f27d8e-da01-43af-8cc5-a1b28dc306d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:00.583715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f27d8e-da01-43af-8cc5-a1b28dc306d5 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:00.594507 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 379/740] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:00.595845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2aa79-4d2a-4e4b-9f77-282ff59456cb None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:00.596136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2aa79-4d2a-4e4b-9f77-282ff59456cb None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:00.596423 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2aa79-4d2a-4e4b-9f77-282ff59456cb None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:00.597967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 380/741] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:00.598678 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2aa79-4d2a-4e4b-9f77-282ff59456cb None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:00.694496 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 362/742] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:03:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 15 10:03:00.695362 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f27d8e-da01-43af-8cc5-a1b28dc306d5 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:00.720879 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2aa79-4d2a-4e4b-9f77-282ff59456cb None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:00.754117 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:00.759281 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615ca7e-a050-4d07-a813-d381ed11f8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:00.759879 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615ca7e-a050-4d07-a813-d381ed11f8d0 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:00.760163 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615ca7e-a050-4d07-a813-d381ed11f8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:00.760435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615ca7e-a050-4d07-a813-d381ed11f8d0 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:00.761211 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615ca7e-a050-4d07-a813-d381ed11f8d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:00.762586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615ca7e-a050-4d07-a813-d381ed11f8d0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:00.786223 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:00.801899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f27d8e-da01-43af-8cc5-a1b28dc306d5 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:00.837624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c936b1-101f-46c2-b4b3-262555d50a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:00.838285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c936b1-101f-46c2-b4b3-262555d50a58 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:00.838564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c936b1-101f-46c2-b4b3-262555d50a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:00.838836 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c936b1-101f-46c2-b4b3-262555d50a58 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:00.839562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c936b1-101f-46c2-b4b3-262555d50a58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:00.839871 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c936b1-101f-46c2-b4b3-262555d50a58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:00.841875 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c936b1-101f-46c2-b4b3-262555d50a58 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:00.853569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 363/743] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:00.854398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c936b1-101f-46c2-b4b3-262555d50a58 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:00.855519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 381/744] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jun 15 10:03:00 2022] POST /identity/v3/services => generated 240 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:03:00.856863 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a4348-8d80-4aec-81de-e71d3d2e2f2b None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:00.857877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a4348-8d80-4aec-81de-e71d3d2e2f2b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:00.858566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a4348-8d80-4aec-81de-e71d3d2e2f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:00.879822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-cb5a4348-8d80-4aec-81de-e71d3d2e2f2b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:00.883358 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-cb5a4348-8d80-4aec-81de-e71d3d2e2f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:00.990118 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 382/745] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:00 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 15 10:03:00.990943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a4348-8d80-4aec-81de-e71d3d2e2f2b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:01.005711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c936b1-101f-46c2-b4b3-262555d50a58 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:01.052236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351b777-5d31-497b-91fc-01477c9f8864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:01.052984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351b777-5d31-497b-91fc-01477c9f8864 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:01.053285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351b777-5d31-497b-91fc-01477c9f8864 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:01.053627 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351b777-5d31-497b-91fc-01477c9f8864 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:01.064373 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351b777-5d31-497b-91fc-01477c9f8864 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:01.065889 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351b777-5d31-497b-91fc-01477c9f8864 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-e351b777-5d31-497b-91fc-01477c9f8864 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:03:01.081117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:01.083135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:01.085685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:01.085685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:01.085685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:03:01.085685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:03:01.085685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:03:01.085685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:03:01.085685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:03:01.085685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:03:01.085685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:03:01.085685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:01.085685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 364/746] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jun 15 10:03:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:01.085685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351b777-5d31-497b-91fc-01477c9f8864 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:01.087348 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:01.124558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a4348-8d80-4aec-81de-e71d3d2e2f2b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:01.153122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:01.174101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7853a-99ed-43cf-9d97-bb908d1ecf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:01.175677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7853a-99ed-43cf-9d97-bb908d1ecf49 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:01.175959 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7853a-99ed-43cf-9d97-bb908d1ecf49 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:01.176233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7853a-99ed-43cf-9d97-bb908d1ecf49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:01.190121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7853a-99ed-43cf-9d97-bb908d1ecf49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:01.192377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7853a-99ed-43cf-9d97-bb908d1ecf49 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:01.202878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351b777-5d31-497b-91fc-01477c9f8864 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:01.218711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-5da7853a-99ed-43cf-9d97-bb908d1ecf49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:01.218711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:01.218711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:01.218711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:01.218711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:01.218711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:01.218711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:01.218711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:01.218711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:01.218711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:01.218711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:01.218711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:01.218711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:01.218711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:01.218711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:01.219768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:01.230175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:01.230175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:01.230175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:01.230175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:01.230175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 383/747] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:01.230175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7853a-99ed-43cf-9d97-bb908d1ecf49 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:01.266438 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b809d-3056-40ad-bef3-2a78422de575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:01.267256 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b809d-3056-40ad-bef3-2a78422de575 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:01.267540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b809d-3056-40ad-bef3-2a78422de575 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:01.267820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b809d-3056-40ad-bef3-2a78422de575 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:01.276192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b809d-3056-40ad-bef3-2a78422de575 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:01.277707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b809d-3056-40ad-bef3-2a78422de575 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-371b809d-3056-40ad-bef3-2a78422de575 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:03:01.293353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:01.294575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:01.296129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:01.296129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:01.296129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:03:01.296129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:03:01.296129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:03:01.296129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:03:01.296129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:03:01.296129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:03:01.296129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:03:01.296129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:01.296129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 365/748] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:03:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:01.296129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b809d-3056-40ad-bef3-2a78422de575 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:01.355067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7853a-99ed-43cf-9d97-bb908d1ecf49 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:01.378058 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:01.394889 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5287a12-3903-4f4e-b6ff-d96f566cafdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:01.395679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5287a12-3903-4f4e-b6ff-d96f566cafdf None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:01.395978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5287a12-3903-4f4e-b6ff-d96f566cafdf None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:01.396260 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5287a12-3903-4f4e-b6ff-d96f566cafdf None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:01.397039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5287a12-3903-4f4e-b6ff-d96f566cafdf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:01.398442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5287a12-3903-4f4e-b6ff-d96f566cafdf None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:01.425533 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b809d-3056-40ad-bef3-2a78422de575 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:01.469889 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 384/749] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jun 15 10:03:01 2022] POST /identity/v3/services => generated 251 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:03:01.470570 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5287a12-3903-4f4e-b6ff-d96f566cafdf None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:01.479313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594f5b8-156f-4eb3-9d1c-2e4488bf2e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:01.480011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594f5b8-156f-4eb3-9d1c-2e4488bf2e34 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:01.480291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594f5b8-156f-4eb3-9d1c-2e4488bf2e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:01.480650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594f5b8-156f-4eb3-9d1c-2e4488bf2e34 None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:01.481505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594f5b8-156f-4eb3-9d1c-2e4488bf2e34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:01.481823 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594f5b8-156f-4eb3-9d1c-2e4488bf2e34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:01.483589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594f5b8-156f-4eb3-9d1c-2e4488bf2e34 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:01.512773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 366/750] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jun 15 10:03:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:01.512773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594f5b8-156f-4eb3-9d1c-2e4488bf2e34 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:01.554621 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:01.660252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5287a12-3903-4f4e-b6ff-d96f566cafdf None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:01.681119 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594f5b8-156f-4eb3-9d1c-2e4488bf2e34 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:01.705041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:01.715776 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f1325-ffa0-4e88-97a9-91680bee780d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:01.717018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f1325-ffa0-4e88-97a9-91680bee780d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:01.717769 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f1325-ffa0-4e88-97a9-91680bee780d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:01.717769 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f1325-ffa0-4e88-97a9-91680bee780d None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:01.718342 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f1325-ffa0-4e88-97a9-91680bee780d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:01.718624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f1325-ffa0-4e88-97a9-91680bee780d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:01.720079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f1325-ffa0-4e88-97a9-91680bee780d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:01.729820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba001bbb-51c9-4cd9-a42c-6ea0b33e728c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:01.730837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba001bbb-51c9-4cd9-a42c-6ea0b33e728c None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:01.731188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba001bbb-51c9-4cd9-a42c-6ea0b33e728c None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:01.731536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba001bbb-51c9-4cd9-a42c-6ea0b33e728c None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:01.733292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba001bbb-51c9-4cd9-a42c-6ea0b33e728c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:01.733676 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba001bbb-51c9-4cd9-a42c-6ea0b33e728c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:01.740023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba001bbb-51c9-4cd9-a42c-6ea0b33e728c None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:01.740220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 385/751] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:01.740220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f1325-ffa0-4e88-97a9-91680bee780d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:01.760244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 367/752] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Jun 15 10:03:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:01.766864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba001bbb-51c9-4cd9-a42c-6ea0b33e728c None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:01.875795 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:01.889596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f1325-ffa0-4e88-97a9-91680bee780d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:01.899997 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba001bbb-51c9-4cd9-a42c-6ea0b33e728c None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:01.937334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f7473-c194-4b96-b774-7f904a3b3121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:01.938701 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f7473-c194-4b96-b774-7f904a3b3121 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:01.938993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f7473-c194-4b96-b774-7f904a3b3121 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:01.939579 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f7473-c194-4b96-b774-7f904a3b3121 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:01.943826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5216bb-503b-41d4-b895-9961d45635e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:01.944719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5216bb-503b-41d4-b895-9961d45635e1 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:01.945121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5216bb-503b-41d4-b895-9961d45635e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:01.945473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5216bb-503b-41d4-b895-9961d45635e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:01.954830 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5216bb-503b-41d4-b895-9961d45635e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:01.955342 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f7473-c194-4b96-b774-7f904a3b3121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:01.956698 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5216bb-503b-41d4-b895-9961d45635e1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:01.956930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f7473-c194-4b96-b774-7f904a3b3121 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:01.974204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-2d5216bb-503b-41d4-b895-9961d45635e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:01.974204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:01.974204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:01.974204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:01.974204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:01.974204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:01.974204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:01.974204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:01.974204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:01.974204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:01.974204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:01.974204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:01.974204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:03:01.974204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:03:01.974204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:01.976027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:01.978560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:03:01.978560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:03:01.978560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:03:01.978560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:03:01.978560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:03:01.978560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:03:01.978560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:01.978560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:01.978560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 368/753] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:01.978560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5216bb-503b-41d4-b895-9961d45635e1 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:01.979904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-659f7473-c194-4b96-b774-7f904a3b3121 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:01.979904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:01.979904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:01.979904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:01.979904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:01.979904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:01.979904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:01.979904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:01.979904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:01.979904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:01.979904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:01.979904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:01.979904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:01.979904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:01.979904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:01.986534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:01.988286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:01.988286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:01.988286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:01.988286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:01.988286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 386/754] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:01.990160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f7473-c194-4b96-b774-7f904a3b3121 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:02.036904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:02.114727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5216bb-503b-41d4-b895-9961d45635e1 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:02.146864 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f7473-c194-4b96-b774-7f904a3b3121 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:02.154760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159310f6-1c5c-4b39-b6e6-3df66c017737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:02.155463 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159310f6-1c5c-4b39-b6e6-3df66c017737 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:02.155752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159310f6-1c5c-4b39-b6e6-3df66c017737 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:02.156033 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159310f6-1c5c-4b39-b6e6-3df66c017737 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:02.187821 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213582e1-eddf-4839-b2fd-59fa6fce9452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:02.188882 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213582e1-eddf-4839-b2fd-59fa6fce9452 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:02.189441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213582e1-eddf-4839-b2fd-59fa6fce9452 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:02.189956 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213582e1-eddf-4839-b2fd-59fa6fce9452 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:02.190983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213582e1-eddf-4839-b2fd-59fa6fce9452 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:02.191536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213582e1-eddf-4839-b2fd-59fa6fce9452 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:02.193730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213582e1-eddf-4839-b2fd-59fa6fce9452 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:02.212577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 387/755] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:02.213976 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213582e1-eddf-4839-b2fd-59fa6fce9452 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:02.218467 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:02.235208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159310f6-1c5c-4b39-b6e6-3df66c017737 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:02.236699 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159310f6-1c5c-4b39-b6e6-3df66c017737 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:02.293529 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:02.335439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213582e1-eddf-4839-b2fd-59fa6fce9452 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:02.355258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 369/756] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:03:01 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:03:02.356590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159310f6-1c5c-4b39-b6e6-3df66c017737 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:02.381785 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2c754-9c36-4d19-965e-6af6d6b59d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:02.382497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2c754-9c36-4d19-965e-6af6d6b59d7a None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:02.382832 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2c754-9c36-4d19-965e-6af6d6b59d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:02.384081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2c754-9c36-4d19-965e-6af6d6b59d7a None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:02.385880 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2c754-9c36-4d19-965e-6af6d6b59d7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:02.386415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2c754-9c36-4d19-965e-6af6d6b59d7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:02.388805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b2c754-9c36-4d19-965e-6af6d6b59d7a None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:02.409051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 388/757] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:03:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:02.410276 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2c754-9c36-4d19-965e-6af6d6b59d7a None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:02.480861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:02.506942 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159310f6-1c5c-4b39-b6e6-3df66c017737 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:02.533012 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2c754-9c36-4d19-965e-6af6d6b59d7a None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:02.561981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc72ed-6825-439c-8c92-0400653721f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:02.563213 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc72ed-6825-439c-8c92-0400653721f2 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:02.563693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc72ed-6825-439c-8c92-0400653721f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:02.564254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc72ed-6825-439c-8c92-0400653721f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:02.567969 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:02.573548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc72ed-6825-439c-8c92-0400653721f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:02.575484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dc72ed-6825-439c-8c92-0400653721f2 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:02.581337 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be273e74-ed52-4e34-a434-b9a3457dfe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:02.582146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be273e74-ed52-4e34-a434-b9a3457dfe8a None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:02.582434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be273e74-ed52-4e34-a434-b9a3457dfe8a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:02.582732 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be273e74-ed52-4e34-a434-b9a3457dfe8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:02.592478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-b6dc72ed-6825-439c-8c92-0400653721f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:02.592478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:02.592478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:02.592478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:02.592478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:02.592478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:02.592478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:02.592478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:02.592478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:02.592478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:02.592478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:02.592478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:02.592478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:03:02.592478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:03:02.592478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:02.593787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:02.594691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:03:02.594691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:03:02.594691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:03:02.594691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:03:02.594691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:03:02.594691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:03:02.594691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:02.594691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:02.595930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be273e74-ed52-4e34-a434-b9a3457dfe8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:02.601954 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be273e74-ed52-4e34-a434-b9a3457dfe8a None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:02.602192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 370/758] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:02.603269 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc72ed-6825-439c-8c92-0400653721f2 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:02.623982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-be273e74-ed52-4e34-a434-b9a3457dfe8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:02.623982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:02.623982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:02.623982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:02.623982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:02.623982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:02.623982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:02.623982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:02.623982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:02.623982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:02.623982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:02.623982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:02.623982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:02.623982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:02.623982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:02.625139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:02.626220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:02.626220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:02.626220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:02.626220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:02.628169 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 389/759] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:02.629563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be273e74-ed52-4e34-a434-b9a3457dfe8a None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:02.725611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dc72ed-6825-439c-8c92-0400653721f2 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:02.765705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44fc3c-90e1-44d5-9354-74b9c4987337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:02.766295 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44fc3c-90e1-44d5-9354-74b9c4987337 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:02.766586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44fc3c-90e1-44d5-9354-74b9c4987337 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:02.766867 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44fc3c-90e1-44d5-9354-74b9c4987337 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:02.767268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be273e74-ed52-4e34-a434-b9a3457dfe8a None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:02.767898 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de44fc3c-90e1-44d5-9354-74b9c4987337 None admin] RBAC: Adding query filter params (role.id=2c276d442f7f4dd59ae7fc3a03943716, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=3b37fadf589c40d28e8df46554d51d01) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:02.768078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de44fc3c-90e1-44d5-9354-74b9c4987337 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:02.769642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de44fc3c-90e1-44d5-9354-74b9c4987337 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:02.776161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 371/760] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:03:02 2022] GET /identity/v3/role_assignments?user.id=3b37fadf589c40d28e8df46554d51d01&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=2c276d442f7f4dd59ae7fc3a03943716 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:02.777043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44fc3c-90e1-44d5-9354-74b9c4987337 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:02.800743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:02.827244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eef663-798d-478a-99df-32a4f8a9e080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:02.828111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eef663-798d-478a-99df-32a4f8a9e080 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:02.829081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eef663-798d-478a-99df-32a4f8a9e080 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:02.829401 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eef663-798d-478a-99df-32a4f8a9e080 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:02.830264 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eef663-798d-478a-99df-32a4f8a9e080 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:02.830589 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eef663-798d-478a-99df-32a4f8a9e080 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:02.832111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eef663-798d-478a-99df-32a4f8a9e080 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:02.846775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 390/761] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:03:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:02.854316 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eef663-798d-478a-99df-32a4f8a9e080 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:02.903304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:02.953444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44fc3c-90e1-44d5-9354-74b9c4987337 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:02.993206 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936417f4-d9ea-417e-b965-1e61d5a47664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:02.993884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936417f4-d9ea-417e-b965-1e61d5a47664 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:02.994169 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936417f4-d9ea-417e-b965-1e61d5a47664 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:02.994446 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936417f4-d9ea-417e-b965-1e61d5a47664 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:02.995183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936417f4-d9ea-417e-b965-1e61d5a47664 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:02.995492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936417f4-d9ea-417e-b965-1e61d5a47664 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:02.996885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936417f4-d9ea-417e-b965-1e61d5a47664 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:03.010781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 372/762] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:03.016454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936417f4-d9ea-417e-b965-1e61d5a47664 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:03.062365 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eef663-798d-478a-99df-32a4f8a9e080 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:03.099872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad441e9-fe0c-4ca7-a001-bdafb352953e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:03.100777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad441e9-fe0c-4ca7-a001-bdafb352953e None admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:03.101095 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad441e9-fe0c-4ca7-a001-bdafb352953e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:03.101376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad441e9-fe0c-4ca7-a001-bdafb352953e None admin] PATH_INFO: `/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/26f23f8595fc43ad94aee7d0192164a1/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:03.105938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad441e9-fe0c-4ca7-a001-bdafb352953e None admin] RBAC: Authorizing `identity:create_grant(project_id=db47f1bb607c4d1a9c346fdc80ecadfa, user_id=26f23f8595fc43ad94aee7d0192164a1, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:03.107972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad441e9-fe0c-4ca7-a001-bdafb352953e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:03.188345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936417f4-d9ea-417e-b965-1e61d5a47664 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:03.198546 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 391/763] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:03:02 2022] PUT /identity/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/26f23f8595fc43ad94aee7d0192164a1/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:03:03.212339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:03.230740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b68cd5-5c3a-441a-b0f0-d7067719e6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:03.231578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b68cd5-5c3a-441a-b0f0-d7067719e6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:03.231913 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b68cd5-5c3a-441a-b0f0-d7067719e6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:03.232686 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b68cd5-5c3a-441a-b0f0-d7067719e6e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:03.241216 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b68cd5-5c3a-441a-b0f0-d7067719e6e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:03.242773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b68cd5-5c3a-441a-b0f0-d7067719e6e0 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:03.259002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-e8b68cd5-5c3a-441a-b0f0-d7067719e6e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:03.259002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:03.259002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:03.259002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:03.259002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:03.259002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:03.259002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:03.259002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:03.259002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:03.259002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:03.259002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:03.259002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:03.259002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:03:03.259002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:03:03.259002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:03.260105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:03.261321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:03:03.261321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:03:03.261321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:03:03.261321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:03:03.261321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:03:03.261321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:03:03.261321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:03.261321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:03.261321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 373/764] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:03.266704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad441e9-fe0c-4ca7-a001-bdafb352953e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:03.337381 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:03.398856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5bcaa-8d64-4d2a-86e0-ff541edfbf3e None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:03.399141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5bcaa-8d64-4d2a-86e0-ff541edfbf3e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:03.399425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5bcaa-8d64-4d2a-86e0-ff541edfbf3e None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:03.401338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 374/765] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:03:03.418653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e442cc-ad23-4482-a5c8-7c7ced9f93e4 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:03.418936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e442cc-ad23-4482-a5c8-7c7ced9f93e4 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:03.419217 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e442cc-ad23-4482-a5c8-7c7ced9f93e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:03.443487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-91e442cc-ad23-4482-a5c8-7c7ced9f93e4 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:03.444734 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-91e442cc-ad23-4482-a5c8-7c7ced9f93e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:03.457079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad441e9-fe0c-4ca7-a001-bdafb352953e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:03.503345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0966082-e978-4e4f-9f86-7c8274283be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:03.504402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0966082-e978-4e4f-9f86-7c8274283be3 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:03.504888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0966082-e978-4e4f-9f86-7c8274283be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:03.505433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0966082-e978-4e4f-9f86-7c8274283be3 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:03.506540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0966082-e978-4e4f-9f86-7c8274283be3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:03.507052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0966082-e978-4e4f-9f86-7c8274283be3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:03.508711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0966082-e978-4e4f-9f86-7c8274283be3 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:03.525847 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 392/766] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:03:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:03.533178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0966082-e978-4e4f-9f86-7c8274283be3 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:03.543538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 375/767] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:03.572799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e442cc-ad23-4482-a5c8-7c7ced9f93e4 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:03.721573 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:03.736879 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0966082-e978-4e4f-9f86-7c8274283be3 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:03.758263 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e442cc-ad23-4482-a5c8-7c7ced9f93e4 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:03.783303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:03.804389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290032a-8ca3-4111-a52a-188dc7a332c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:03.805089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290032a-8ca3-4111-a52a-188dc7a332c6 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:03.805375 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290032a-8ca3-4111-a52a-188dc7a332c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:03.805658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290032a-8ca3-4111-a52a-188dc7a332c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:03.806519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290032a-8ca3-4111-a52a-188dc7a332c6 None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=589e66c6228a44adbc9e418794d7d91b) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:03.806835 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290032a-8ca3-4111-a52a-188dc7a332c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:03.808574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290032a-8ca3-4111-a52a-188dc7a332c6 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:03.819282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 393/768] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:03:03 2022] GET /identity/v3/role_assignments?user.id=589e66c6228a44adbc9e418794d7d91b&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:03.837265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85e041-589f-4618-8376-6710cf32207a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:03.838103 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85e041-589f-4618-8376-6710cf32207a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:03.838390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85e041-589f-4618-8376-6710cf32207a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:03.838668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85e041-589f-4618-8376-6710cf32207a None admin] PATH_INFO: `/v3/services/network` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:03.839348 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85e041-589f-4618-8376-6710cf32207a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:03.840874 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85e041-589f-4618-8376-6710cf32207a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-1c85e041-589f-4618-8376-6710cf32207a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:03.860739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:03.861814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:03.862654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:03.862654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:03.862654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:03.862654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:03.862654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:03.862654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:03.862654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:03.862654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:03.862654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:03.862654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:03.862654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:03.862654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:03.862654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:03:03.862654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:03.867363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 376/769] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:03 2022] GET /identity/v3/services/network => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:03.868979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290032a-8ca3-4111-a52a-188dc7a332c6 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:03.933711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cfcaf-8113-40c3-88ae-4657961d168e None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:03.934027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cfcaf-8113-40c3-88ae-4657961d168e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:03.934276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cfcaf-8113-40c3-88ae-4657961d168e None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:03.935943 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 377/770] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:03.951700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2b1ba-e83e-4a85-b0b1-eff70139eb38 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:03.952063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2b1ba-e83e-4a85-b0b1-eff70139eb38 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:03.952261 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2b1ba-e83e-4a85-b0b1-eff70139eb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:03.977015 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-59a2b1ba-e83e-4a85-b0b1-eff70139eb38 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:03.978251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-59a2b1ba-e83e-4a85-b0b1-eff70139eb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:04.084016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290032a-8ca3-4111-a52a-188dc7a332c6 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:04.109079 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 378/771] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:04.134223 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2b1ba-e83e-4a85-b0b1-eff70139eb38 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:04.154773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823f78f-52c2-4344-9fb6-a0f0f9cc5892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:04.155454 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823f78f-52c2-4344-9fb6-a0f0f9cc5892 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:04.155739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823f78f-52c2-4344-9fb6-a0f0f9cc5892 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:04.156237 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823f78f-52c2-4344-9fb6-a0f0f9cc5892 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:04.156729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823f78f-52c2-4344-9fb6-a0f0f9cc5892 None admin] RBAC: Adding query filter params (name=network) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:04.157063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823f78f-52c2-4344-9fb6-a0f0f9cc5892 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:04.158450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823f78f-52c2-4344-9fb6-a0f0f9cc5892 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:04.174242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 394/772] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jun 15 10:03:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:04.181751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823f78f-52c2-4344-9fb6-a0f0f9cc5892 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:04.211022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:04.287207 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2b1ba-e83e-4a85-b0b1-eff70139eb38 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:04.289912 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:04.320491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823f78f-52c2-4344-9fb6-a0f0f9cc5892 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:04.349204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6157f-9762-4919-bc7c-2958c730cc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:04.349995 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6157f-9762-4919-bc7c-2958c730cc54 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:04.350324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6157f-9762-4919-bc7c-2958c730cc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:04.350653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6157f-9762-4919-bc7c-2958c730cc54 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:04.351383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6157f-9762-4919-bc7c-2958c730cc54 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:04.352983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6157f-9762-4919-bc7c-2958c730cc54 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-96a6157f-9762-4919-bc7c-2958c730cc54 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:04.386043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:04.427705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:04.428531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 379/773] 10.222.0.60 () {56 vars in 1129 bytes} [Wed Jun 15 10:03:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:04.429389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6157f-9762-4919-bc7c-2958c730cc54 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:04.429469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da231504-6001-4012-9d1d-6f4294cc4599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:04.429469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da231504-6001-4012-9d1d-6f4294cc4599 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:04.429469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da231504-6001-4012-9d1d-6f4294cc4599 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:04.429469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da231504-6001-4012-9d1d-6f4294cc4599 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:04.429725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da231504-6001-4012-9d1d-6f4294cc4599 None admin] RBAC: Adding query filter params (type=network) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:04.429725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da231504-6001-4012-9d1d-6f4294cc4599 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:04.429725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da231504-6001-4012-9d1d-6f4294cc4599 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:04.429725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 395/774] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jun 15 10:03:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:04.429725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da231504-6001-4012-9d1d-6f4294cc4599 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:04.565354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:04.598948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da231504-6001-4012-9d1d-6f4294cc4599 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:04.608407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6157f-9762-4919-bc7c-2958c730cc54 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:04.667795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e370c-e00d-4548-affd-eb98f5f274ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:04.668730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e370c-e00d-4548-affd-eb98f5f274ee None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:04.669187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e370c-e00d-4548-affd-eb98f5f274ee None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:04.670048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1db9d0-0a68-4181-9415-a6c20ccd19ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:04.670795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e370c-e00d-4548-affd-eb98f5f274ee None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:04.671242 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e370c-e00d-4548-affd-eb98f5f274ee None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:04.671663 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e370c-e00d-4548-affd-eb98f5f274ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:04.673299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e370c-e00d-4548-affd-eb98f5f274ee None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:04.673429 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1db9d0-0a68-4181-9415-a6c20ccd19ae None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:04.673845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1db9d0-0a68-4181-9415-a6c20ccd19ae None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:04.674266 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1db9d0-0a68-4181-9415-a6c20ccd19ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:04.675873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1db9d0-0a68-4181-9415-a6c20ccd19ae None admin] RBAC: Adding query filter params (interface=public, service_id=803d3bc9258e47d7beebc01c0b04e578, region_id=RegionOne) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:04.676771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1db9d0-0a68-4181-9415-a6c20ccd19ae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:04.684980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1db9d0-0a68-4181-9415-a6c20ccd19ae None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:04.685142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 380/775] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Jun 15 10:03:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:04.685876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e370c-e00d-4548-affd-eb98f5f274ee None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:04.702903 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 396/776] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jun 15 10:03:04 2022] GET /identity/v3/endpoints?service_id=803d3bc9258e47d7beebc01c0b04e578&interface=public®ion_id=RegionOne => generated 194 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:04.707920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1db9d0-0a68-4181-9415-a6c20ccd19ae None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:04.743712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:04.889466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e370c-e00d-4548-affd-eb98f5f274ee None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:04.930442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1db9d0-0a68-4181-9415-a6c20ccd19ae None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:04.961919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e094c37-30fe-4fdd-acee-ce52147a8ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:04.963127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e094c37-30fe-4fdd-acee-ce52147a8ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:04.964767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e094c37-30fe-4fdd-acee-ce52147a8ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:04.966012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e094c37-30fe-4fdd-acee-ce52147a8ba2 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:04.967554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e094c37-30fe-4fdd-acee-ce52147a8ba2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:04.969418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e094c37-30fe-4fdd-acee-ce52147a8ba2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:04.970828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e094c37-30fe-4fdd-acee-ce52147a8ba2 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:04.986474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf7d98-d480-43d3-80a3-b97fd6026df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:04.987470 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf7d98-d480-43d3-80a3-b97fd6026df6 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:04.987578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 381/777] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Jun 15 10:03:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:04.988049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf7d98-d480-43d3-80a3-b97fd6026df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:04.988541 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf7d98-d480-43d3-80a3-b97fd6026df6 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:04.989469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf7d98-d480-43d3-80a3-b97fd6026df6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:04.991029 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf7d98-d480-43d3-80a3-b97fd6026df6 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:04.992897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e094c37-30fe-4fdd-acee-ce52147a8ba2 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:04.996938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 397/778] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jun 15 10:03:04 2022] GET /identity/v3/services => generated 772 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:05.133203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:05.153296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e094c37-30fe-4fdd-acee-ce52147a8ba2 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:05.198384 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fbc16-2537-4b84-890a-a4bb8bb2e14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:05.199083 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fbc16-2537-4b84-890a-a4bb8bb2e14f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:05.199369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fbc16-2537-4b84-890a-a4bb8bb2e14f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:05.205410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fbc16-2537-4b84-890a-a4bb8bb2e14f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:05.205410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fbc16-2537-4b84-890a-a4bb8bb2e14f None admin] RBAC: Adding query filter params (interface=public, service_id=572252b7af824e92aaf2a2711f7558a5, region_id=RegionOne) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:05.205410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fbc16-2537-4b84-890a-a4bb8bb2e14f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:05.205410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fbc16-2537-4b84-890a-a4bb8bb2e14f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:05.217917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 382/779] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jun 15 10:03:04 2022] GET /identity/v3/endpoints?service_id=572252b7af824e92aaf2a2711f7558a5&interface=public®ion_id=RegionOne => generated 194 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:05.223354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf7d98-d480-43d3-80a3-b97fd6026df6 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:05.255829 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:05.357616 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf7d98-d480-43d3-80a3-b97fd6026df6 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:05.400057 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f8f033-78b2-4f3c-90c0-8c39eac78ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:05.400758 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8f033-78b2-4f3c-90c0-8c39eac78ccc None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:05.401076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8f033-78b2-4f3c-90c0-8c39eac78ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:05.401357 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f8f033-78b2-4f3c-90c0-8c39eac78ccc None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:05.402072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8f033-78b2-4f3c-90c0-8c39eac78ccc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:05.404116 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f8f033-78b2-4f3c-90c0-8c39eac78ccc None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:05.417205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 398/780] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jun 15 10:03:05 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:05.456290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a42f5d-1511-4604-ad11-3704f334ea26 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:05.456669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a42f5d-1511-4604-ad11-3704f334ea26 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:05.456852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a42f5d-1511-4604-ad11-3704f334ea26 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:05.458575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 383/781] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:05.464494 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e409d-8061-4a76-92c7-38472dfe35f8 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:05.464807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e409d-8061-4a76-92c7-38472dfe35f8 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:05.465132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e409d-8061-4a76-92c7-38472dfe35f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:05.484675 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-907e409d-8061-4a76-92c7-38472dfe35f8 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:05.485934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-907e409d-8061-4a76-92c7-38472dfe35f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:05.595699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 399/782] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:05.624918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a42f5d-1511-4604-ad11-3704f334ea26 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:05.739158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a42f5d-1511-4604-ad11-3704f334ea26 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:05.776934 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181f511-cff4-4eb0-bf82-6bc28a567693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:05.777777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181f511-cff4-4eb0-bf82-6bc28a567693 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:05.778125 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181f511-cff4-4eb0-bf82-6bc28a567693 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:05.778403 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181f511-cff4-4eb0-bf82-6bc28a567693 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:05.791020 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181f511-cff4-4eb0-bf82-6bc28a567693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-8181f511-cff4-4eb0-bf82-6bc28a567693 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:03:05.800156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:05.801402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:05.802226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:05.802226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:05.802226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:05.802226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:05.802226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:03:05.802226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:03:05.802226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:03:05.802226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:03:05.802226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:03:05.802226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:03:05.802226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:03:05.802226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:05.802226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 384/783] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:03:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:05.808190 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e409d-8061-4a76-92c7-38472dfe35f8 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:05.822121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:05.938850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e409d-8061-4a76-92c7-38472dfe35f8 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:05.991198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d7862-1a51-4e46-a58c-9f8af714b767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:05.991802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d7862-1a51-4e46-a58c-9f8af714b767 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:05.992090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d7862-1a51-4e46-a58c-9f8af714b767 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:05.992368 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d7862-1a51-4e46-a58c-9f8af714b767 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:05.993176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d7862-1a51-4e46-a58c-9f8af714b767 None admin] RBAC: Adding query filter params (name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:05.993488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d7862-1a51-4e46-a58c-9f8af714b767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:05.994884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d7862-1a51-4e46-a58c-9f8af714b767 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:05.998044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbb0b9-55a1-4519-87d4-8562a9eb5feb None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:05.999210 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbb0b9-55a1-4519-87d4-8562a9eb5feb None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:05.999210 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbb0b9-55a1-4519-87d4-8562a9eb5feb None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:06.000003 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 385/784] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:06.005547 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d9950-36c3-46dc-9b9e-0ec4e3d28c46 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:06.005838 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d9950-36c3-46dc-9b9e-0ec4e3d28c46 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:06.006123 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d9950-36c3-46dc-9b9e-0ec4e3d28c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:06.008647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 400/785] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:03:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:06.014357 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d7862-1a51-4e46-a58c-9f8af714b767 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:06.036245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-aa1d9950-36c3-46dc-9b9e-0ec4e3d28c46 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:06.037494 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-aa1d9950-36c3-46dc-9b9e-0ec4e3d28c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:06.070153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:06.143004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:06.176929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d7862-1a51-4e46-a58c-9f8af714b767 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:06.189190 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 386/786] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:06.215534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28244b2-8c57-48f1-a82d-f34be707ef9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:06.216910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28244b2-8c57-48f1-a82d-f34be707ef9c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:06.217276 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28244b2-8c57-48f1-a82d-f34be707ef9c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:06.217562 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28244b2-8c57-48f1-a82d-f34be707ef9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:06.227951 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d9950-36c3-46dc-9b9e-0ec4e3d28c46 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:06.232579 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28244b2-8c57-48f1-a82d-f34be707ef9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:06.234099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28244b2-8c57-48f1-a82d-f34be707ef9c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:06.258455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-c28244b2-8c57-48f1-a82d-f34be707ef9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:06.258455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:06.258455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:06.258455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:06.258455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:06.258455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:06.258455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:06.258455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:06.258455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:06.258455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:06.258455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:06.258455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:06.258455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:06.258455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:06.258455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:06.259363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:06.260259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:06.260259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:06.260259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:06.260259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:06.260259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 401/787] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:06.267596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28244b2-8c57-48f1-a82d-f34be707ef9c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:06.382176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28244b2-8c57-48f1-a82d-f34be707ef9c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:06.409063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:06.423271 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57fa6f-dd9e-427e-99d8-6f4dba931ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:06.424433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57fa6f-dd9e-427e-99d8-6f4dba931ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:06.424924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57fa6f-dd9e-427e-99d8-6f4dba931ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:06.425707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57fa6f-dd9e-427e-99d8-6f4dba931ef7 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:06.426694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57fa6f-dd9e-427e-99d8-6f4dba931ef7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:06.427211 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57fa6f-dd9e-427e-99d8-6f4dba931ef7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:06.428860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57fa6f-dd9e-427e-99d8-6f4dba931ef7 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:06.449519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 402/788] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:06.451768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57fa6f-dd9e-427e-99d8-6f4dba931ef7 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:06.527491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d9950-36c3-46dc-9b9e-0ec4e3d28c46 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:06.579945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08952135-aa83-4673-ae37-875c4482a0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:06.581527 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08952135-aa83-4673-ae37-875c4482a0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:06.582449 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08952135-aa83-4673-ae37-875c4482a0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:06.583238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08952135-aa83-4673-ae37-875c4482a0f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:06.588586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57fa6f-dd9e-427e-99d8-6f4dba931ef7 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:06.598651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08952135-aa83-4673-ae37-875c4482a0f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-08952135-aa83-4673-ae37-875c4482a0f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:03:06.609899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:06.610883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:06.611743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:06.611743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:06.611743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:06.611743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:06.611743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:03:06.611743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:03:06.611743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:03:06.611743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:03:06.611743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:03:06.611743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:03:06.611743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:03:06.611743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:06.615560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 387/789] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:03:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:06.620318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60d210-a9b0-4b1e-a573-d347c541c626 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:06.620318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60d210-a9b0-4b1e-a573-d347c541c626 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:06.620318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60d210-a9b0-4b1e-a573-d347c541c626 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:06.625165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 388/790] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:06.626931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60d210-a9b0-4b1e-a573-d347c541c626 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:06.643576 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:06.644792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c68e42-1d92-40dd-a81f-d578a3b4f4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:06.646090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c68e42-1d92-40dd-a81f-d578a3b4f4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:06.646448 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c68e42-1d92-40dd-a81f-d578a3b4f4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:06.646793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c68e42-1d92-40dd-a81f-d578a3b4f4e9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:06.657979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c68e42-1d92-40dd-a81f-d578a3b4f4e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:06.659868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c68e42-1d92-40dd-a81f-d578a3b4f4e9 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-79c68e42-1d92-40dd-a81f-d578a3b4f4e9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:03:06.681584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:06.682698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:06.683638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:06.683638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:06.683638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:03:06.683638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:03:06.683638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:03:06.683638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:03:06.683638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:03:06.683638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:03:06.683638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:03:06.683638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:06.688839 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 403/791] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jun 15 10:03:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:06.690731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9045c6-bf99-46ad-baa6-5d150afc6325 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:06.691021 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9045c6-bf99-46ad-baa6-5d150afc6325 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:06.691300 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9045c6-bf99-46ad-baa6-5d150afc6325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:06.715574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-1f9045c6-bf99-46ad-baa6-5d150afc6325 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:06.717869 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-1f9045c6-bf99-46ad-baa6-5d150afc6325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:06.767186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60d210-a9b0-4b1e-a573-d347c541c626 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:06.770248 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:06.813060 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 404/792] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:06.813991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9045c6-bf99-46ad-baa6-5d150afc6325 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:06.815604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960fd5d-9f86-488d-981b-4459c976951c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:06.816718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960fd5d-9f86-488d-981b-4459c976951c None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:06.818079 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960fd5d-9f86-488d-981b-4459c976951c None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:06.818593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960fd5d-9f86-488d-981b-4459c976951c None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:06.819584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960fd5d-9f86-488d-981b-4459c976951c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:06.820116 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960fd5d-9f86-488d-981b-4459c976951c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:06.821906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960fd5d-9f86-488d-981b-4459c976951c None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:06.839589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 389/793] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:03:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:06.843080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960fd5d-9f86-488d-981b-4459c976951c None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:06.945333 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:06.951689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9045c6-bf99-46ad-baa6-5d150afc6325 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:06.976715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960fd5d-9f86-488d-981b-4459c976951c None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:06.995584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326f30d-8c23-4608-8875-8b621bf6da4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:06.996696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326f30d-8c23-4608-8875-8b621bf6da4e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:06.997985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326f30d-8c23-4608-8875-8b621bf6da4e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:06.998610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326f30d-8c23-4608-8875-8b621bf6da4e None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:07.000866 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326f30d-8c23-4608-8875-8b621bf6da4e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:07.001439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326f30d-8c23-4608-8875-8b621bf6da4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:07.003145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b326f30d-8c23-4608-8875-8b621bf6da4e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:07.027043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c9a97-d1a4-4e7d-b824-5233a67f544a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:07.027043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c9a97-d1a4-4e7d-b824-5233a67f544a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:07.027043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c9a97-d1a4-4e7d-b824-5233a67f544a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:07.027043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c9a97-d1a4-4e7d-b824-5233a67f544a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:07.028226 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 405/794] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Jun 15 10:03:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:07.029911 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326f30d-8c23-4608-8875-8b621bf6da4e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:07.038082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3c9a97-d1a4-4e7d-b824-5233a67f544a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-cb3c9a97-d1a4-4e7d-b824-5233a67f544a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:03:07.047187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:07.048308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:07.049367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:07.049367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:07.049367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:07.049367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:07.049367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:03:07.049367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:03:07.049367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:03:07.049367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:03:07.049367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:03:07.049367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:03:07.049367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:03:07.049367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:07.056233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 390/795] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:03:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:07.056233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c9a97-d1a4-4e7d-b824-5233a67f544a None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:07.074049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:07.161130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b326f30d-8c23-4608-8875-8b621bf6da4e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:07.174635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3c9a97-d1a4-4e7d-b824-5233a67f544a None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:07.203415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f56a43-9810-4187-aee1-6b82c48523c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:07.204591 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f56a43-9810-4187-aee1-6b82c48523c3 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:07.205138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f56a43-9810-4187-aee1-6b82c48523c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:07.205714 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f56a43-9810-4187-aee1-6b82c48523c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:07.206271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:07.211451 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9280f4-1158-4e74-9837-0664e04cc249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:07.212318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9280f4-1158-4e74-9837-0664e04cc249 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:07.212611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9280f4-1158-4e74-9837-0664e04cc249 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:07.212926 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9280f4-1158-4e74-9837-0664e04cc249 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:07.220435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f56a43-9810-4187-aee1-6b82c48523c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:07.222390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f56a43-9810-4187-aee1-6b82c48523c3 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:07.224291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9280f4-1158-4e74-9837-0664e04cc249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:07.229545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9280f4-1158-4e74-9837-0664e04cc249 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:07.243212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-c7f56a43-9810-4187-aee1-6b82c48523c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:07.243212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:07.243212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:07.243212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:07.243212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:07.243212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:07.243212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:07.243212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:07.243212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:07.243212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:07.243212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:07.243212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:07.243212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:07.243212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:07.243212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:07.244135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:07.245059 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:07.245059 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:07.245059 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:07.245059 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:07.245276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-ee9280f4-1158-4e74-9837-0664e04cc249 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:07.245276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:07.245276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:07.245276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:07.245276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:07.245276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:07.245276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:07.245276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:07.245276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:07.245276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:07.245276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:07.245276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:07.245276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:07.245276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:07.245276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:07.246095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:07.248395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:07.248395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:07.248395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:07.248395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:07.251256 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 391/797] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:07.251490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 406/797] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:07.252233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9280f4-1158-4e74-9837-0664e04cc249 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:07.253995 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f56a43-9810-4187-aee1-6b82c48523c3 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:07.351442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:07.358904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9280f4-1158-4e74-9837-0664e04cc249 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:07.375184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f56a43-9810-4187-aee1-6b82c48523c3 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:07.395825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616104b-dd09-4ecc-bfef-8e8fcd426d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:07.396559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616104b-dd09-4ecc-bfef-8e8fcd426d9c None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:07.397003 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616104b-dd09-4ecc-bfef-8e8fcd426d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:07.397309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616104b-dd09-4ecc-bfef-8e8fcd426d9c None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:07.398084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616104b-dd09-4ecc-bfef-8e8fcd426d9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:07.398397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616104b-dd09-4ecc-bfef-8e8fcd426d9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:07.399828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616104b-dd09-4ecc-bfef-8e8fcd426d9c None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:07.413121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 392/798] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:03:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:07.413121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616104b-dd09-4ecc-bfef-8e8fcd426d9c None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:07.426547 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228ae5c-1211-467e-b1a8-6df020457a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:07.427404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228ae5c-1211-467e-b1a8-6df020457a7f None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:07.427787 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228ae5c-1211-467e-b1a8-6df020457a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:07.428171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228ae5c-1211-467e-b1a8-6df020457a7f None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:07.429247 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228ae5c-1211-467e-b1a8-6df020457a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:07.429661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228ae5c-1211-467e-b1a8-6df020457a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:07.431332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228ae5c-1211-467e-b1a8-6df020457a7f None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:07.449509 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 407/799] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:07.449509 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228ae5c-1211-467e-b1a8-6df020457a7f None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:07.478818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:07.548137 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616104b-dd09-4ecc-bfef-8e8fcd426d9c None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:07.594846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62106f20-4310-4b68-b222-9c7df3c89c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:07.595696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228ae5c-1211-467e-b1a8-6df020457a7f None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:07.596132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62106f20-4310-4b68-b222-9c7df3c89c3e None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:07.596568 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62106f20-4310-4b68-b222-9c7df3c89c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:07.597006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62106f20-4310-4b68-b222-9c7df3c89c3e None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:07.598065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62106f20-4310-4b68-b222-9c7df3c89c3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:07.598513 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62106f20-4310-4b68-b222-9c7df3c89c3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:07.600895 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62106f20-4310-4b68-b222-9c7df3c89c3e None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:07.618703 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 393/800] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:07.620306 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62106f20-4310-4b68-b222-9c7df3c89c3e None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:07.635213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f0549-6480-47b2-a4fe-e4f4d6fa33eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:07.636009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f0549-6480-47b2-a4fe-e4f4d6fa33eb None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:07.636396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f0549-6480-47b2-a4fe-e4f4d6fa33eb None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:07.636779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f0549-6480-47b2-a4fe-e4f4d6fa33eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:07.649398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f0549-6480-47b2-a4fe-e4f4d6fa33eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:07.650850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f0549-6480-47b2-a4fe-e4f4d6fa33eb None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:07.652944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:07.673201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-4b2f0549-6480-47b2-a4fe-e4f4d6fa33eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:07.673201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:07.673201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:07.673201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:07.673201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:07.673201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:07.673201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:07.673201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:07.673201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:07.673201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:07.673201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:07.673201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:07.673201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:07.673201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:07.673201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:07.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:07.675097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:07.675097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:07.675097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:07.675097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:07.682660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 408/801] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:07.683963 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f0549-6480-47b2-a4fe-e4f4d6fa33eb None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:07.755823 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62106f20-4310-4b68-b222-9c7df3c89c3e None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:07.767936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:07.805954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e4ec9-6244-4633-bcbb-33ea4c809b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:07.806765 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e4ec9-6244-4633-bcbb-33ea4c809b5f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:07.807057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e4ec9-6244-4633-bcbb-33ea4c809b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:07.807338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e4ec9-6244-4633-bcbb-33ea4c809b5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:07.814711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f0549-6480-47b2-a4fe-e4f4d6fa33eb None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:07.815327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e4ec9-6244-4633-bcbb-33ea4c809b5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:07.816795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e4ec9-6244-4633-bcbb-33ea4c809b5f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:07.831574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-794e4ec9-6244-4633-bcbb-33ea4c809b5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:07.831574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:07.831574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:07.831574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:07.831574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:07.831574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:07.831574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:07.831574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:07.831574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:07.831574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:07.831574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:07.831574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:07.831574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:03:07.831574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:03:07.831574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:07.832500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:07.833389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:03:07.833389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:03:07.833389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:03:07.833389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:03:07.833389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:03:07.833389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:03:07.833389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:07.833389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:07.833389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 394/802] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:07.833941 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e4ec9-6244-4633-bcbb-33ea4c809b5f None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:07.855848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60ce57-fb7a-43bd-bd8e-2e7ff4ad6a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:07.857342 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60ce57-fb7a-43bd-bd8e-2e7ff4ad6a10 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:07.857644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60ce57-fb7a-43bd-bd8e-2e7ff4ad6a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:07.857922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60ce57-fb7a-43bd-bd8e-2e7ff4ad6a10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:07.866482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60ce57-fb7a-43bd-bd8e-2e7ff4ad6a10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:07.867957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60ce57-fb7a-43bd-bd8e-2e7ff4ad6a10 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-2d60ce57-fb7a-43bd-bd8e-2e7ff4ad6a10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:03:07.882820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:07.883747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:07.884667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:07.884667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:07.884667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:03:07.884667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:03:07.884667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:03:07.884667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:03:07.884667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:03:07.884667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:03:07.884667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:03:07.884667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:07.889220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 409/803] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jun 15 10:03:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:07.890724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9478c-72e4-4d25-9a7d-8495600adb9b None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:07.891019 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9478c-72e4-4d25-9a7d-8495600adb9b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:07.891297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9478c-72e4-4d25-9a7d-8495600adb9b None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:07.892844 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 410/804] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:07.893912 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9478c-72e4-4d25-9a7d-8495600adb9b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:07.959623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e4ec9-6244-4633-bcbb-33ea4c809b5f None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:07.961369 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:07.998045 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8254f-d1cb-4a44-8015-c0513b72aa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:07.998768 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8254f-d1cb-4a44-8015-c0513b72aa93 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:07.999058 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8254f-d1cb-4a44-8015-c0513b72aa93 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:07.999339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8254f-d1cb-4a44-8015-c0513b72aa93 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:08.000075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8254f-d1cb-4a44-8015-c0513b72aa93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:08.000390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8254f-d1cb-4a44-8015-c0513b72aa93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:08.001811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8254f-d1cb-4a44-8015-c0513b72aa93 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:08.013963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 395/805] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:08.014263 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9478c-72e4-4d25-9a7d-8495600adb9b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:08.014819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8254f-d1cb-4a44-8015-c0513b72aa93 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:08.043398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:08.053483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e083356-46b3-4816-83c3-9ce32f823a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:08.054148 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e083356-46b3-4816-83c3-9ce32f823a36 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:08.054440 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e083356-46b3-4816-83c3-9ce32f823a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:08.054738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e083356-46b3-4816-83c3-9ce32f823a36 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:08.061925 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e083356-46b3-4816-83c3-9ce32f823a36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:08.061925 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e083356-46b3-4816-83c3-9ce32f823a36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:08.061925 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e083356-46b3-4816-83c3-9ce32f823a36 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:08.070202 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 411/806] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:03:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:08.071467 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57386d-8eb8-41b3-9e7c-15567241425d None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:08.071758 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57386d-8eb8-41b3-9e7c-15567241425d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:08.072037 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57386d-8eb8-41b3-9e7c-15567241425d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:08.088225 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-9f57386d-8eb8-41b3-9e7c-15567241425d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:08.089458 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9f57386d-8eb8-41b3-9e7c-15567241425d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:08.117119 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8254f-d1cb-4a44-8015-c0513b72aa93 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:08.169604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d1654-857e-4b0d-b5e4-345f52a9dc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:08.169604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d1654-857e-4b0d-b5e4-345f52a9dc8d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:08.169604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d1654-857e-4b0d-b5e4-345f52a9dc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:08.169604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d1654-857e-4b0d-b5e4-345f52a9dc8d None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:08.170004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d1654-857e-4b0d-b5e4-345f52a9dc8d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:08.170004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d1654-857e-4b0d-b5e4-345f52a9dc8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:08.170258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 412/807] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:08.170355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d1654-857e-4b0d-b5e4-345f52a9dc8d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:08.170436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8ba70-25d2-4837-b827-bc547cce0fb3 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:08.170436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8ba70-25d2-4837-b827-bc547cce0fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:08.170436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8ba70-25d2-4837-b827-bc547cce0fb3 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:08.170436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 413/808] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:08.170436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8ba70-25d2-4837-b827-bc547cce0fb3 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:08.182291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 396/809] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jun 15 10:03:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:08.183105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d1654-857e-4b0d-b5e4-345f52a9dc8d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:08.213867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:08.268809 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:08.269455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8ba70-25d2-4837-b827-bc547cce0fb3 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:08.284774 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d1654-857e-4b0d-b5e4-345f52a9dc8d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:08.303809 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac86c8-5811-4901-a1be-8f6c2191a52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:08.304613 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac86c8-5811-4901-a1be-8f6c2191a52e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:08.304901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac86c8-5811-4901-a1be-8f6c2191a52e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:08.305214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac86c8-5811-4901-a1be-8f6c2191a52e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:08.313216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac86c8-5811-4901-a1be-8f6c2191a52e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:08.314865 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac86c8-5811-4901-a1be-8f6c2191a52e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:08.319484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3154a-e620-4295-b79f-163ec03ad1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:08.320375 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3154a-e620-4295-b79f-163ec03ad1aa None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:08.320656 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3154a-e620-4295-b79f-163ec03ad1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:08.320928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3154a-e620-4295-b79f-163ec03ad1aa None admin] PATH_INFO: `/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/3b37fadf589c40d28e8df46554d51d01/roles/2c276d442f7f4dd59ae7fc3a03943716` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-83ac86c8-5811-4901-a1be-8f6c2191a52e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:03:08.330242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:08.331398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:08.332488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:08.332488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:08.332488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:03:08.332488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:03:08.332488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:03:08.332488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:03:08.332488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:03:08.332488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:03:08.332488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:03:08.332488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:08.332488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 414/810] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:03:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:08.333103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee806ec4-6647-468e-a127-0738d77d0240 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:08.333103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee806ec4-6647-468e-a127-0738d77d0240 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:08.333221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee806ec4-6647-468e-a127-0738d77d0240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:08.337168 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3154a-e620-4295-b79f-163ec03ad1aa None admin] RBAC: Authorizing `identity:create_grant(project_id=db47f1bb607c4d1a9c346fdc80ecadfa, user_id=3b37fadf589c40d28e8df46554d51d01, role_id=2c276d442f7f4dd59ae7fc3a03943716)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:08.338829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3154a-e620-4295-b79f-163ec03ad1aa None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:08.350089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-ee806ec4-6647-468e-a127-0738d77d0240 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:08.351279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ee806ec4-6647-468e-a127-0738d77d0240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:08.417637 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 415/811] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:08.418502 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee806ec4-6647-468e-a127-0738d77d0240 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:08.447841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 397/812] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:03:08 2022] PUT /identity/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/3b37fadf589c40d28e8df46554d51d01/roles/2c276d442f7f4dd59ae7fc3a03943716 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:03:08.448594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3154a-e620-4295-b79f-163ec03ad1aa None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:08.456295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:08.535220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee806ec4-6647-468e-a127-0738d77d0240 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:08.578799 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97ac6a-f0f3-4e69-a0b5-f084fa161c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:08.579624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97ac6a-f0f3-4e69-a0b5-f084fa161c17 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:08.579912 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97ac6a-f0f3-4e69-a0b5-f084fa161c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:08.580196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97ac6a-f0f3-4e69-a0b5-f084fa161c17 None admin] PATH_INFO: `/v3/services/network` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:08.580423 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3154a-e620-4295-b79f-163ec03ad1aa None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:08.580876 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97ac6a-f0f3-4e69-a0b5-f084fa161c17 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:08.582353 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97ac6a-f0f3-4e69-a0b5-f084fa161c17 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-1d97ac6a-f0f3-4e69-a0b5-f084fa161c17 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:08.598107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:08.599361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:08.600198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:08.600198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:08.600198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:08.600198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:08.600198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:08.600198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:08.600198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:08.600198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:08.600198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:08.600198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:08.600198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:08.600198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:08.600198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 15 10:03:08.600198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:08.600198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 416/813] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:08 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:08.601008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97ac6a-f0f3-4e69-a0b5-f084fa161c17 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:08.628318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c21cd-ada4-4b88-a797-348c2addc71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:08.629096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c21cd-ada4-4b88-a797-348c2addc71d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:08.629442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c21cd-ada4-4b88-a797-348c2addc71d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:08.629782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c21cd-ada4-4b88-a797-348c2addc71d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:08.633291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:08.706296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c21cd-ada4-4b88-a797-348c2addc71d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:08.708388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93c21cd-ada4-4b88-a797-348c2addc71d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:08.717013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:08.721400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97ac6a-f0f3-4e69-a0b5-f084fa161c17 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:08.758654 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3bc34-f19f-4e72-b386-e31405fa0da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:08.759338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3bc34-f19f-4e72-b386-e31405fa0da6 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:08.759619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3bc34-f19f-4e72-b386-e31405fa0da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:08.759903 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3bc34-f19f-4e72-b386-e31405fa0da6 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:08.760752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3bc34-f19f-4e72-b386-e31405fa0da6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:08.761193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3bc34-f19f-4e72-b386-e31405fa0da6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:08.762793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3bc34-f19f-4e72-b386-e31405fa0da6 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:08.782400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 417/814] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Jun 15 10:03:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:08.783385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3bc34-f19f-4e72-b386-e31405fa0da6 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:08.820820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 398/815] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:03:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:03:08.822026 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c21cd-ada4-4b88-a797-348c2addc71d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:08.858032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:08.896355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3bc34-f19f-4e72-b386-e31405fa0da6 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:08.936045 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d9acb-286a-41e3-be1a-a2ef071bc944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:08.936982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d9acb-286a-41e3-be1a-a2ef071bc944 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:08.937272 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d9acb-286a-41e3-be1a-a2ef071bc944 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:08.937553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d9acb-286a-41e3-be1a-a2ef071bc944 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:08.938220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d9acb-286a-41e3-be1a-a2ef071bc944 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:08.939611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d9acb-286a-41e3-be1a-a2ef071bc944 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:08.940848 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93c21cd-ada4-4b88-a797-348c2addc71d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-188d9acb-286a-41e3-be1a-a2ef071bc944 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:08.952638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:08.953807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:08.954901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:08.954901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:08.954901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:08.954901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:08.954901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:08.954901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:08.954901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:08.954901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:08.954901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:08.954901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:08.954901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:08.954901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:08.954901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:08.954901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 15 10:03:08.954901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:08.958884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 418/816] 10.222.0.60 () {56 vars in 1129 bytes} [Wed Jun 15 10:03:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:08.959712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d9acb-286a-41e3-be1a-a2ef071bc944 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:08.983442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a338e3-0d6d-4f5f-a853-3fd9db309f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:08.984121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a338e3-0d6d-4f5f-a853-3fd9db309f76 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:08.984414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a338e3-0d6d-4f5f-a853-3fd9db309f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:08.984925 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:08.985397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a338e3-0d6d-4f5f-a853-3fd9db309f76 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:08.985950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a338e3-0d6d-4f5f-a853-3fd9db309f76 None admin] RBAC: Adding query filter params (name=network) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:08.986869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a338e3-0d6d-4f5f-a853-3fd9db309f76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:08.988294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a338e3-0d6d-4f5f-a853-3fd9db309f76 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:08.993711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 399/817] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jun 15 10:03:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:08.994543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a338e3-0d6d-4f5f-a853-3fd9db309f76 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:09.074802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d9acb-286a-41e3-be1a-a2ef071bc944 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:09.100508 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:09.104443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a338e3-0d6d-4f5f-a853-3fd9db309f76 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:09.113447 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26548d4-caaa-4d4b-a249-aaa374b71291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:09.114402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26548d4-caaa-4d4b-a249-aaa374b71291 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:09.114793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26548d4-caaa-4d4b-a249-aaa374b71291 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:09.115146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26548d4-caaa-4d4b-a249-aaa374b71291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:09.127410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26548d4-caaa-4d4b-a249-aaa374b71291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:09.129112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26548d4-caaa-4d4b-a249-aaa374b71291 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:09.140387 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c0999-31fb-4c02-ae77-edf1a4125e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:09.141290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c0999-31fb-4c02-ae77-edf1a4125e85 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:09.141649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c0999-31fb-4c02-ae77-edf1a4125e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:09.141999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c0999-31fb-4c02-ae77-edf1a4125e85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:09.148740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-f26548d4-caaa-4d4b-a249-aaa374b71291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:09.148740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:09.148740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:09.148740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:09.148740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:09.148740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:09.148740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:09.148740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:09.148740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:09.148740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:09.148740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:09.148740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:09.148740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:09.148740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:09.148740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:09.149679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:09.156371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:09.156371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:09.156371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:09.156371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:09.156371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 419/818] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:09.156371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26548d4-caaa-4d4b-a249-aaa374b71291 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:09.156760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c0999-31fb-4c02-ae77-edf1a4125e85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:09.156760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c0999-31fb-4c02-ae77-edf1a4125e85 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:09.179567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-183c0999-31fb-4c02-ae77-edf1a4125e85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:09.179567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:09.179567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:09.179567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:09.179567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:09.179567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:09.179567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:09.179567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:09.179567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:09.179567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:09.179567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:09.179567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:09.179567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:09.179567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:09.179567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:09.180651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:09.186773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:09.186773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:09.186773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:09.186773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:09.186773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 400/819] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:09.186773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c0999-31fb-4c02-ae77-edf1a4125e85 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:09.240062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:09.270985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26548d4-caaa-4d4b-a249-aaa374b71291 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:09.295389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c0999-31fb-4c02-ae77-edf1a4125e85 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:09.309630 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae016bc9-df2f-4665-9c48-2b21de080217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:09.311068 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae016bc9-df2f-4665-9c48-2b21de080217 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:09.311404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae016bc9-df2f-4665-9c48-2b21de080217 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:09.311752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae016bc9-df2f-4665-9c48-2b21de080217 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:09.312548 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae016bc9-df2f-4665-9c48-2b21de080217 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:09.312940 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae016bc9-df2f-4665-9c48-2b21de080217 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:09.314425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae016bc9-df2f-4665-9c48-2b21de080217 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:09.323998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 420/820] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Jun 15 10:03:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:09.323998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae016bc9-df2f-4665-9c48-2b21de080217 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:09.338822 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f28816-d530-4bcd-b0d4-963c4896358f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:09.339703 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f28816-d530-4bcd-b0d4-963c4896358f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:09.340060 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f28816-d530-4bcd-b0d4-963c4896358f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:09.340402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f28816-d530-4bcd-b0d4-963c4896358f None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:09.341780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f28816-d530-4bcd-b0d4-963c4896358f None admin] RBAC: Adding query filter params (type=network) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:09.342153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f28816-d530-4bcd-b0d4-963c4896358f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:09.343775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f28816-d530-4bcd-b0d4-963c4896358f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:09.353544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 401/821] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jun 15 10:03:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:09.353544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f28816-d530-4bcd-b0d4-963c4896358f None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:09.368564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:09.437375 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae016bc9-df2f-4665-9c48-2b21de080217 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:09.465436 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f28816-d530-4bcd-b0d4-963c4896358f None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:09.473938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3e669-7e95-40f6-906b-21162b534863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:09.474606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3e669-7e95-40f6-906b-21162b534863 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:09.474894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3e669-7e95-40f6-906b-21162b534863 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:09.475175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3e669-7e95-40f6-906b-21162b534863 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:09.476359 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3e669-7e95-40f6-906b-21162b534863 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:09.476673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3e669-7e95-40f6-906b-21162b534863 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:09.478479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3e669-7e95-40f6-906b-21162b534863 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:09.481306 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:09.495891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 421/822] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:09.495891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3e669-7e95-40f6-906b-21162b534863 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:09.510646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cfda8-1e8e-4ffa-b886-4682bba460bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:09.511322 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cfda8-1e8e-4ffa-b886-4682bba460bb None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:09.511603 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cfda8-1e8e-4ffa-b886-4682bba460bb None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:09.511884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cfda8-1e8e-4ffa-b886-4682bba460bb None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:09.512624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cfda8-1e8e-4ffa-b886-4682bba460bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:09.513540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cfda8-1e8e-4ffa-b886-4682bba460bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:09.514918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0cfda8-1e8e-4ffa-b886-4682bba460bb None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:09.531842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 402/823] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:09.532749 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cfda8-1e8e-4ffa-b886-4682bba460bb None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:09.611206 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3e669-7e95-40f6-906b-21162b534863 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:09.614143 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:09.643746 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0cfda8-1e8e-4ffa-b886-4682bba460bb None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:09.653230 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fbb92-2465-4e69-8838-664cd37232ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:09.654692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fbb92-2465-4e69-8838-664cd37232ef None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:09.655041 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fbb92-2465-4e69-8838-664cd37232ef None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:09.655393 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fbb92-2465-4e69-8838-664cd37232ef None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:09.656190 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fbb92-2465-4e69-8838-664cd37232ef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:09.656571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fbb92-2465-4e69-8838-664cd37232ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:09.658060 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fbb92-2465-4e69-8838-664cd37232ef None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:09.663572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 422/824] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Jun 15 10:03:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:09.664717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fbb92-2465-4e69-8838-664cd37232ef None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:09.685629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e3590-079f-4b13-97d9-9ce6ddd79a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:09.686312 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e3590-079f-4b13-97d9-9ce6ddd79a21 None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:09.686598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e3590-079f-4b13-97d9-9ce6ddd79a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:09.686875 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e3590-079f-4b13-97d9-9ce6ddd79a21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:09.687633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e3590-079f-4b13-97d9-9ce6ddd79a21 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:09.689184 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e3590-079f-4b13-97d9-9ce6ddd79a21 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:09.746295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:09.787197 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fbb92-2465-4e69-8838-664cd37232ef None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:09.798310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 403/825] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jun 15 10:03:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:03:09.806520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e3590-079f-4b13-97d9-9ce6ddd79a21 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:09.841503 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5716b99-fc5d-4434-b005-e5de79216b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:09.842386 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5716b99-fc5d-4434-b005-e5de79216b86 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:09.842670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5716b99-fc5d-4434-b005-e5de79216b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:09.842947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5716b99-fc5d-4434-b005-e5de79216b86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:09.855308 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5716b99-fc5d-4434-b005-e5de79216b86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:09.858165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5716b99-fc5d-4434-b005-e5de79216b86 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:09.878888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-f5716b99-fc5d-4434-b005-e5de79216b86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:09.878888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:09.878888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:09.878888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:09.878888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:09.878888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:09.878888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:09.878888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:09.878888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:09.878888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:09.878888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:09.878888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:09.878888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:03:09.878888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:03:09.878888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:09.880024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:09.881111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:03:09.881111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:03:09.881111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:03:09.881111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:03:09.881111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:03:09.881111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:03:09.881111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:09.881111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:09.887413 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 423/826] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:09.887413 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5716b99-fc5d-4434-b005-e5de79216b86 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:09.947725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e3590-079f-4b13-97d9-9ce6ddd79a21 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:09.982382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:09.990314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6b357-021a-4dbb-89e6-f313a1e081d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:09.991932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6b357-021a-4dbb-89e6-f313a1e081d7 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:09.992328 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6b357-021a-4dbb-89e6-f313a1e081d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:09.992707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6b357-021a-4dbb-89e6-f313a1e081d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:10.002828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6b357-021a-4dbb-89e6-f313a1e081d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:10.004230 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6b357-021a-4dbb-89e6-f313a1e081d7 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:10.012010 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5716b99-fc5d-4434-b005-e5de79216b86 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:10.029466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-0ec6b357-021a-4dbb-89e6-f313a1e081d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:10.029466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:10.029466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:10.029466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:10.029466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:10.029466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:10.029466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:10.029466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:10.029466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:10.029466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:10.029466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:10.029466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:10.029466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:03:10.029466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:03:10.029466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:10.030414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:10.031481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:03:10.031481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:03:10.031481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:03:10.031481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:03:10.031481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:03:10.031481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:03:10.031481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:10.031481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:10.031481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 404/827] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:10.031955 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6b357-021a-4dbb-89e6-f313a1e081d7 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:10.039031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:10.065311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7b028-8119-4f50-b6ba-e824c389c42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:10.066820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7b028-8119-4f50-b6ba-e824c389c42b None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:10.066820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7b028-8119-4f50-b6ba-e824c389c42b None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:10.066820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7b028-8119-4f50-b6ba-e824c389c42b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:10.067333 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7b028-8119-4f50-b6ba-e824c389c42b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:10.068729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7b028-8119-4f50-b6ba-e824c389c42b None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:10.151292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6b357-021a-4dbb-89e6-f313a1e081d7 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:10.156990 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 424/828] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jun 15 10:03:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:03:10.157850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7b028-8119-4f50-b6ba-e824c389c42b None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:10.208020 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3995d82-4924-4db0-a536-9461bd3da554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:10.208988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3995d82-4924-4db0-a536-9461bd3da554 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:10.209551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3995d82-4924-4db0-a536-9461bd3da554 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:10.209974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3995d82-4924-4db0-a536-9461bd3da554 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:10.211002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3995d82-4924-4db0-a536-9461bd3da554 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:10.211466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3995d82-4924-4db0-a536-9461bd3da554 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:10.213268 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3995d82-4924-4db0-a536-9461bd3da554 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:10.234649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 405/829] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:03:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:10.236450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3995d82-4924-4db0-a536-9461bd3da554 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:10.293697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:10.301187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7b028-8119-4f50-b6ba-e824c389c42b None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:10.342382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2194b-5adf-48fe-b9f3-c9eb8846a83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:10.343072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2194b-5adf-48fe-b9f3-c9eb8846a83e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:10.343361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2194b-5adf-48fe-b9f3-c9eb8846a83e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:10.343641 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2194b-5adf-48fe-b9f3-c9eb8846a83e None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:10.345387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2194b-5adf-48fe-b9f3-c9eb8846a83e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:10.345710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2194b-5adf-48fe-b9f3-c9eb8846a83e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:10.347158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2194b-5adf-48fe-b9f3-c9eb8846a83e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:10.362474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 425/830] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:03:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:10.368707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2194b-5adf-48fe-b9f3-c9eb8846a83e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:10.376719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3995d82-4924-4db0-a536-9461bd3da554 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:10.409325 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:10.422466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45268c70-b057-4568-a00e-cc7616ee88fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:10.423830 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45268c70-b057-4568-a00e-cc7616ee88fd None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:10.424123 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45268c70-b057-4568-a00e-cc7616ee88fd None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:10.424405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45268c70-b057-4568-a00e-cc7616ee88fd None admin] PATH_INFO: `/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/589e66c6228a44adbc9e418794d7d91b/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:10.429390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45268c70-b057-4568-a00e-cc7616ee88fd None admin] RBAC: Authorizing `identity:create_grant(project_id=db47f1bb607c4d1a9c346fdc80ecadfa, user_id=589e66c6228a44adbc9e418794d7d91b, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:10.430861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45268c70-b057-4568-a00e-cc7616ee88fd None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:10.493554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2194b-5adf-48fe-b9f3-c9eb8846a83e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:10.501936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 406/831] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:03:10 2022] PUT /identity/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/589e66c6228a44adbc9e418794d7d91b/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:03:10.545992 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf02cf-9167-4194-9182-4522b7f3f470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:10.546944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf02cf-9167-4194-9182-4522b7f3f470 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:10.547393 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf02cf-9167-4194-9182-4522b7f3f470 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:10.547819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf02cf-9167-4194-9182-4522b7f3f470 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:10.549020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf02cf-9167-4194-9182-4522b7f3f470 None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=26f23f8595fc43ad94aee7d0192164a1) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:10.549500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf02cf-9167-4194-9182-4522b7f3f470 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:10.551402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf02cf-9167-4194-9182-4522b7f3f470 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:10.561027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 426/832] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:03:10 2022] GET /identity/v3/role_assignments?user.id=26f23f8595fc43ad94aee7d0192164a1&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:10.706727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811e571-3a91-454f-a1d7-fb77ffffaa63 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:10.707205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811e571-3a91-454f-a1d7-fb77ffffaa63 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:10.707674 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811e571-3a91-454f-a1d7-fb77ffffaa63 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:10.709665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 407/833] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:10.715469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323b2bc-8749-4b59-883a-bcd1ee3015fb None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:10.716626 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323b2bc-8749-4b59-883a-bcd1ee3015fb None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:10.717058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323b2bc-8749-4b59-883a-bcd1ee3015fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:10.738103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-b323b2bc-8749-4b59-883a-bcd1ee3015fb None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:10.739629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b323b2bc-8749-4b59-883a-bcd1ee3015fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:10.817042 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 427/834] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:10.840724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811e571-3a91-454f-a1d7-fb77ffffaa63 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:10.959428 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811e571-3a91-454f-a1d7-fb77ffffaa63 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:10.967330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:10.996483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb1076-67c1-4a44-ba23-dbec6cb5c53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:10.998795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb1076-67c1-4a44-ba23-dbec6cb5c53e None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:10.999247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb1076-67c1-4a44-ba23-dbec6cb5c53e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:10.999665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb1076-67c1-4a44-ba23-dbec6cb5c53e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:11.013049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb1076-67c1-4a44-ba23-dbec6cb5c53e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-fbdb1076-67c1-4a44-ba23-dbec6cb5c53e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:03:11.021504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:11.029204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:11.030292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:11.030292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:11.030292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:11.030292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:11.030292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:03:11.030292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:03:11.030292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:03:11.030292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:03:11.030292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:03:11.030292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:03:11.030292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:03:11.030292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:11.030292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 408/835] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:03:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:11.032443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323b2bc-8749-4b59-883a-bcd1ee3015fb None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:11.064481 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:11.272306 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b323b2bc-8749-4b59-883a-bcd1ee3015fb None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:11.308515 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a27fdd-649f-4492-9e31-914fa2d8e37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:11.309218 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27fdd-649f-4492-9e31-914fa2d8e37f None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:11.309533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27fdd-649f-4492-9e31-914fa2d8e37f None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:11.310310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27fdd-649f-4492-9e31-914fa2d8e37f None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:11.311068 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a27fdd-649f-4492-9e31-914fa2d8e37f None admin] RBAC: Adding query filter params (name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:11.311537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a27fdd-649f-4492-9e31-914fa2d8e37f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:11.313307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a27fdd-649f-4492-9e31-914fa2d8e37f None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:11.322311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 428/836] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:03:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:11.329062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb1076-67c1-4a44-ba23-dbec6cb5c53e None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:11.441825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb1076-67c1-4a44-ba23-dbec6cb5c53e None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:11.477717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2720e-c663-4866-85cf-46c0a3ba42b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:11.478899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2720e-c663-4866-85cf-46c0a3ba42b1 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:11.479380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2720e-c663-4866-85cf-46c0a3ba42b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:11.479846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2720e-c663-4866-85cf-46c0a3ba42b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:11.492460 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2720e-c663-4866-85cf-46c0a3ba42b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:11.494253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2720e-c663-4866-85cf-46c0a3ba42b1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:11.513862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-23c2720e-c663-4866-85cf-46c0a3ba42b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:11.513862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:11.513862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:11.513862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:11.513862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:11.513862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:11.513862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:11.513862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:11.513862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:11.513862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:11.513862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:11.513862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:11.513862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:11.513862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:11.513862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:11.514864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:11.515732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:11.515732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:11.515732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:11.515732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:11.518018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 409/837] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:11.523798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a27fdd-649f-4492-9e31-914fa2d8e37f None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:11.618531 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:11.635765 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a27fdd-649f-4492-9e31-914fa2d8e37f None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:11.671543 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14585bf-c761-4eff-a778-262ae059c82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:11.672291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14585bf-c761-4eff-a778-262ae059c82c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:11.672578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14585bf-c761-4eff-a778-262ae059c82c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:11.672858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14585bf-c761-4eff-a778-262ae059c82c None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:11.673632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14585bf-c761-4eff-a778-262ae059c82c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:11.673943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14585bf-c761-4eff-a778-262ae059c82c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:11.675793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14585bf-c761-4eff-a778-262ae059c82c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:11.689665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 429/838] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:11.695262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2720e-c663-4866-85cf-46c0a3ba42b1 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:11.842559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:11.930403 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2720e-c663-4866-85cf-46c0a3ba42b1 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:11.966599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041d407-7615-46ad-bf28-4fb8df3b941b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:11.967520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041d407-7615-46ad-bf28-4fb8df3b941b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:11.967803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041d407-7615-46ad-bf28-4fb8df3b941b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:11.968076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041d407-7615-46ad-bf28-4fb8df3b941b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:11.975558 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041d407-7615-46ad-bf28-4fb8df3b941b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:11.977143 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041d407-7615-46ad-bf28-4fb8df3b941b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-4041d407-7615-46ad-bf28-4fb8df3b941b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:03:11.993244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:11.994202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:11.995126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:11.995126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:11.995126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:03:11.995126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:03:11.995126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:03:11.995126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:03:11.995126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:03:11.995126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:03:11.995126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:03:11.995126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:11.995126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 410/839] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jun 15 10:03:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:12.001233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14585bf-c761-4eff-a778-262ae059c82c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:12.107845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14585bf-c761-4eff-a778-262ae059c82c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:12.142135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff504c-cb02-479c-b13b-177d62044dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:12.142812 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff504c-cb02-479c-b13b-177d62044dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:12.143201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff504c-cb02-479c-b13b-177d62044dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:12.143378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff504c-cb02-479c-b13b-177d62044dc2 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:12.144257 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff504c-cb02-479c-b13b-177d62044dc2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:12.144613 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff504c-cb02-479c-b13b-177d62044dc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:12.146083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff504c-cb02-479c-b13b-177d62044dc2 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:12.168630 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 430/840] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Jun 15 10:03:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:12.174710 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041d407-7615-46ad-bf28-4fb8df3b941b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:12.260400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:12.283466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041d407-7615-46ad-bf28-4fb8df3b941b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:12.320029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8c121-8d6d-40b1-a03d-fce30a06f6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:12.320683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8c121-8d6d-40b1-a03d-fce30a06f6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:12.320983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8c121-8d6d-40b1-a03d-fce30a06f6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:12.321259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8c121-8d6d-40b1-a03d-fce30a06f6a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:12.394348 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8c121-8d6d-40b1-a03d-fce30a06f6a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:12.395724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8c121-8d6d-40b1-a03d-fce30a06f6a1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:12.489348 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:12.508670 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 411/841] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:03:12 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:03:12.514020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd71df-f484-4399-a3b2-132cb0d3454e None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:12.514393 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd71df-f484-4399-a3b2-132cb0d3454e None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:12.514684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd71df-f484-4399-a3b2-132cb0d3454e None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:12.514866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8c121-8d6d-40b1-a03d-fce30a06f6a1 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:12.516327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 431/842] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:12.534835 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00345b6e-d1f7-4ed0-96c5-55bb6b66850d None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:12.535741 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00345b6e-d1f7-4ed0-96c5-55bb6b66850d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:12.536432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00345b6e-d1f7-4ed0-96c5-55bb6b66850d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:12.562712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:12.573699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-00345b6e-d1f7-4ed0-96c5-55bb6b66850d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:12.577310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-00345b6e-d1f7-4ed0-96c5-55bb6b66850d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:12.626493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8c121-8d6d-40b1-a03d-fce30a06f6a1 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:12.667030 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13c6b4-e34e-4cd3-a639-260092e8a153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:12.667743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13c6b4-e34e-4cd3-a639-260092e8a153 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:12.668018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13c6b4-e34e-4cd3-a639-260092e8a153 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:12.668285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13c6b4-e34e-4cd3-a639-260092e8a153 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:12.681151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13c6b4-e34e-4cd3-a639-260092e8a153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:12.682581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13c6b4-e34e-4cd3-a639-260092e8a153 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:12.702489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-3c13c6b4-e34e-4cd3-a639-260092e8a153 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:12.702489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:12.702489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:12.702489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:12.702489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:12.702489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:12.702489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:12.702489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:12.702489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:12.702489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:12.702489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:12.702489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:12.702489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:12.702489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:12.702489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:12.703448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:12.704314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:12.704314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:12.704314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:12.704314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:12.711424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 412/844] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:12.711424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13c6b4-e34e-4cd3-a639-260092e8a153 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:12.712029 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 432/844] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:12.733316 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00345b6e-d1f7-4ed0-96c5-55bb6b66850d None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:12.746581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:12.822100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13c6b4-e34e-4cd3-a639-260092e8a153 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:12.844717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00345b6e-d1f7-4ed0-96c5-55bb6b66850d None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:12.858673 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a2559-2833-4357-bcbd-66944d9b597f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:12.859338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a2559-2833-4357-bcbd-66944d9b597f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:12.859616 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a2559-2833-4357-bcbd-66944d9b597f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:12.859887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a2559-2833-4357-bcbd-66944d9b597f None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:12.860620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a2559-2833-4357-bcbd-66944d9b597f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:12.860925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a2559-2833-4357-bcbd-66944d9b597f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:12.862408 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a2559-2833-4357-bcbd-66944d9b597f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:12.879347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 413/845] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:12.879347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ca667-e159-43d4-8945-6a3ca9145baf None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:12.879347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ca667-e159-43d4-8945-6a3ca9145baf None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:12.879347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ca667-e159-43d4-8945-6a3ca9145baf None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:12.879347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 414/846] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:12.880228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ca667-e159-43d4-8945-6a3ca9145baf None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:12.887966 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d5418-8e2d-45d0-bb80-22558753afb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:12.888873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d5418-8e2d-45d0-bb80-22558753afb1 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:12.889192 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d5418-8e2d-45d0-bb80-22558753afb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:12.889467 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d5418-8e2d-45d0-bb80-22558753afb1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:12.890154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d5418-8e2d-45d0-bb80-22558753afb1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:12.891525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d5418-8e2d-45d0-bb80-22558753afb1 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:12.894242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-582d5418-8e2d-45d0-bb80-22558753afb1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:12.905030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:12.912011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:12.913061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:12.913061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:12.913061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:12.913061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:12.913061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:12.913061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:12.913061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:12.913061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:12.913061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:12.913061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:12.913061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:12.913061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:12.913061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:03:12.913061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:12.913061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 433/847] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:03:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:12.913999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3f4a2-29d8-4c40-8712-284a289266c8 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:12.913999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3f4a2-29d8-4c40-8712-284a289266c8 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:12.913999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3f4a2-29d8-4c40-8712-284a289266c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:12.927570 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-b0a3f4a2-29d8-4c40-8712-284a289266c8 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:12.928698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b0a3f4a2-29d8-4c40-8712-284a289266c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:12.991008 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:13.003574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ca667-e159-43d4-8945-6a3ca9145baf None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:13.004191 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 434/848] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:13.004944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3f4a2-29d8-4c40-8712-284a289266c8 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:13.052169 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a9fdb-e0ff-4a99-94a6-6dde3d1a90d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:13.052999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a9fdb-e0ff-4a99-94a6-6dde3d1a90d0 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:13.053289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a9fdb-e0ff-4a99-94a6-6dde3d1a90d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:13.053573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a9fdb-e0ff-4a99-94a6-6dde3d1a90d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:13.061548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a9fdb-e0ff-4a99-94a6-6dde3d1a90d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:13.063006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a9fdb-e0ff-4a99-94a6-6dde3d1a90d0 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:13.084598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-3b2a9fdb-e0ff-4a99-94a6-6dde3d1a90d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:13.084598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:13.084598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:13.084598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:13.084598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:13.084598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:13.084598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:13.084598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:13.084598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:13.084598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:13.084598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:13.084598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:13.084598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:03:13.084598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:03:13.084598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:13.085650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:13.086532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:03:13.086532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:03:13.086532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:03:13.086532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:03:13.086532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:03:13.086532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:03:13.086532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:13.086532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:13.086958 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 415/849] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:13.088095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb2fbf-5699-4474-b0e1-3aa8bd9c3d54 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:13.088388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb2fbf-5699-4474-b0e1-3aa8bd9c3d54 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:13.088725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb2fbf-5699-4474-b0e1-3aa8bd9c3d54 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:13.091668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 416/850] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:13.091668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb2fbf-5699-4474-b0e1-3aa8bd9c3d54 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:13.113260 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a3f4a2-29d8-4c40-8712-284a289266c8 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:13.142157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:13.147567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5aa30-06a4-4fd8-9301-3323f70efcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:13.148226 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5aa30-06a4-4fd8-9301-3323f70efcb5 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:13.148513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5aa30-06a4-4fd8-9301-3323f70efcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:13.155267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5aa30-06a4-4fd8-9301-3323f70efcb5 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:13.155267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5aa30-06a4-4fd8-9301-3323f70efcb5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:13.155267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5aa30-06a4-4fd8-9301-3323f70efcb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:13.155267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5aa30-06a4-4fd8-9301-3323f70efcb5 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:13.157119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 435/851] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:03:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:13.157983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5aa30-06a4-4fd8-9301-3323f70efcb5 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:13.195001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb2fbf-5699-4474-b0e1-3aa8bd9c3d54 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:13.227900 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49de825-3737-4cb4-926f-4939db13783b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:13.228713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49de825-3737-4cb4-926f-4939db13783b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:13.229024 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49de825-3737-4cb4-926f-4939db13783b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:13.229308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49de825-3737-4cb4-926f-4939db13783b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:13.241120 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49de825-3737-4cb4-926f-4939db13783b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-e49de825-3737-4cb4-926f-4939db13783b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:03:13.249864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:13.250946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:13.251811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:13.251811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:13.251811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:13.251811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:13.251811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:03:13.251811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:03:13.251811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:03:13.251811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:03:13.251811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:03:13.251811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:03:13.251811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:03:13.251811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:13.251811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 417/852] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:03:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:13.252499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a8081-df04-4df2-8938-500ffe2cc9d0 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:13.252499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a8081-df04-4df2-8938-500ffe2cc9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:13.252820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a8081-df04-4df2-8938-500ffe2cc9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:13.263349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5aa30-06a4-4fd8-9301-3323f70efcb5 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:13.269159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-875a8081-df04-4df2-8938-500ffe2cc9d0 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:13.270389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-875a8081-df04-4df2-8938-500ffe2cc9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:13.273257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:13.297344 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd30eab-11ad-4207-bce9-1878fb5af4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:13.298008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd30eab-11ad-4207-bce9-1878fb5af4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:13.298297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd30eab-11ad-4207-bce9-1878fb5af4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:13.298581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd30eab-11ad-4207-bce9-1878fb5af4c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:13.299376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd30eab-11ad-4207-bce9-1878fb5af4c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:13.299691 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd30eab-11ad-4207-bce9-1878fb5af4c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:13.301426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd30eab-11ad-4207-bce9-1878fb5af4c7 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:13.314221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 436/853] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:03:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:13.315354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd30eab-11ad-4207-bce9-1878fb5af4c7 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:13.342390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 418/854] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:13.343441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a8081-df04-4df2-8938-500ffe2cc9d0 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:13.385103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:13.431002 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd30eab-11ad-4207-bce9-1878fb5af4c7 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:13.444016 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a8081-df04-4df2-8938-500ffe2cc9d0 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:13.466274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780db6a-72ff-43ab-b650-d1f0685cc8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:13.467170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780db6a-72ff-43ab-b650-d1f0685cc8bf None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:13.467462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780db6a-72ff-43ab-b650-d1f0685cc8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:13.467773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780db6a-72ff-43ab-b650-d1f0685cc8bf None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:13.468488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780db6a-72ff-43ab-b650-d1f0685cc8bf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:13.468796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780db6a-72ff-43ab-b650-d1f0685cc8bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:13.470217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780db6a-72ff-43ab-b650-d1f0685cc8bf None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:13.474667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 437/855] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:03:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:13.475710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780db6a-72ff-43ab-b650-d1f0685cc8bf None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:13.477043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becc1b3-f2f8-41b0-be62-3e47e2dd73b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:13.477730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becc1b3-f2f8-41b0-be62-3e47e2dd73b2 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:13.478017 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becc1b3-f2f8-41b0-be62-3e47e2dd73b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:13.478297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becc1b3-f2f8-41b0-be62-3e47e2dd73b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:13.479078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7becc1b3-f2f8-41b0-be62-3e47e2dd73b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:13.479388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7becc1b3-f2f8-41b0-be62-3e47e2dd73b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:13.480835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7becc1b3-f2f8-41b0-be62-3e47e2dd73b2 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:13.499940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 419/856] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:03:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:13.499940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becc1b3-f2f8-41b0-be62-3e47e2dd73b2 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:13.501209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:13.603576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780db6a-72ff-43ab-b650-d1f0685cc8bf None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:13.624797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:13.627471 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becc1b3-f2f8-41b0-be62-3e47e2dd73b2 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:13.642432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c8ae2-64c4-4b57-9e9d-9118b69202f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:13.643092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c8ae2-64c4-4b57-9e9d-9118b69202f3 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:13.643374 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c8ae2-64c4-4b57-9e9d-9118b69202f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:13.643655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c8ae2-64c4-4b57-9e9d-9118b69202f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:13.644527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c8ae2-64c4-4b57-9e9d-9118b69202f3 None admin] RBAC: Adding query filter params (role.id=2c276d442f7f4dd59ae7fc3a03943716, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=3b37fadf589c40d28e8df46554d51d01) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:13.644846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c8ae2-64c4-4b57-9e9d-9118b69202f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:13.646348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c8ae2-64c4-4b57-9e9d-9118b69202f3 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:13.654223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 438/857] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:03:13 2022] GET /identity/v3/role_assignments?user.id=3b37fadf589c40d28e8df46554d51d01&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=2c276d442f7f4dd59ae7fc3a03943716 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:13.654748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c8ae2-64c4-4b57-9e9d-9118b69202f3 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:13.682151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703e84f-75c0-4a87-856f-102c05a98810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:13.682998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703e84f-75c0-4a87-856f-102c05a98810 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:13.683283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703e84f-75c0-4a87-856f-102c05a98810 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:13.683558 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703e84f-75c0-4a87-856f-102c05a98810 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:13.684613 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703e84f-75c0-4a87-856f-102c05a98810 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:13.685769 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703e84f-75c0-4a87-856f-102c05a98810 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-9703e84f-75c0-4a87-856f-102c05a98810 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:13.704038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:13.705275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:13.706282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:13.706282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:13.706282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:13.706282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:13.706282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:13.706282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:13.706282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:13.706282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:13.706282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:13.706282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:13.706282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:13.706282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:13.706282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:13.706282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:03:13.706282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:13.709945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 420/858] 10.222.0.60 () {56 vars in 1132 bytes} [Wed Jun 15 10:03:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:13.711841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703e84f-75c0-4a87-856f-102c05a98810 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:13.793245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c8ae2-64c4-4b57-9e9d-9118b69202f3 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:13.806830 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:13.847217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f191aa-f91b-41a2-b492-72a73f840660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:13.848327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f191aa-f91b-41a2-b492-72a73f840660 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:13.848612 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f191aa-f91b-41a2-b492-72a73f840660 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:13.848883 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f191aa-f91b-41a2-b492-72a73f840660 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:13.857650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703e84f-75c0-4a87-856f-102c05a98810 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:13.863592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f191aa-f91b-41a2-b492-72a73f840660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:13.865081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f191aa-f91b-41a2-b492-72a73f840660 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:13.887484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-e7f191aa-f91b-41a2-b492-72a73f840660 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:13.887484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:13.887484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:13.887484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:13.887484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:13.887484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:13.887484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:13.887484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:13.887484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:13.887484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:13.887484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:13.887484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:13.887484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:13.887484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:13.887484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:13.889372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:13.890486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:13.890486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:13.890486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:13.890486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:13.890486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 439/859] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:13.896563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f191aa-f91b-41a2-b492-72a73f840660 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:13.907004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858ac10-d46e-4d00-b060-d10970423be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:13.907677 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858ac10-d46e-4d00-b060-d10970423be3 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:13.907976 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858ac10-d46e-4d00-b060-d10970423be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:13.908245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858ac10-d46e-4d00-b060-d10970423be3 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:13.908988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858ac10-d46e-4d00-b060-d10970423be3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:13.909293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858ac10-d46e-4d00-b060-d10970423be3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:13.910655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858ac10-d46e-4d00-b060-d10970423be3 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:13.916676 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 421/860] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jun 15 10:03:13 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:13.922221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858ac10-d46e-4d00-b060-d10970423be3 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:13.944308 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:14.037329 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f191aa-f91b-41a2-b492-72a73f840660 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:14.045939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858ac10-d46e-4d00-b060-d10970423be3 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:14.078556 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:14.084976 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c62c7-260e-40e9-a582-92021b5d07c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:14.085841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c62c7-260e-40e9-a582-92021b5d07c3 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:14.090244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c62c7-260e-40e9-a582-92021b5d07c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:14.090244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c62c7-260e-40e9-a582-92021b5d07c3 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:14.090244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c62c7-260e-40e9-a582-92021b5d07c3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:14.090244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c62c7-260e-40e9-a582-92021b5d07c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:14.090561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccfbf92-68ac-46cf-bfb5-6ebfef21c76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:14.090761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c62c7-260e-40e9-a582-92021b5d07c3 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:14.091329 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfbf92-68ac-46cf-bfb5-6ebfef21c76c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:14.091329 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfbf92-68ac-46cf-bfb5-6ebfef21c76c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:14.091329 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccfbf92-68ac-46cf-bfb5-6ebfef21c76c None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:14.091329 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccfbf92-68ac-46cf-bfb5-6ebfef21c76c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:14.091666 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccfbf92-68ac-46cf-bfb5-6ebfef21c76c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:14.093847 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccfbf92-68ac-46cf-bfb5-6ebfef21c76c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:14.095542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 422/861] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jun 15 10:03:13 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:14.108105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 440/862] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:14.114143 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c62c7-260e-40e9-a582-92021b5d07c3 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:14.238660 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c62c7-260e-40e9-a582-92021b5d07c3 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:14.280613 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d357637-4d9b-4f9f-acd8-e86686fbf702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:14.282596 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d357637-4d9b-4f9f-acd8-e86686fbf702 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:14.282596 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d357637-4d9b-4f9f-acd8-e86686fbf702 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:14.282596 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d357637-4d9b-4f9f-acd8-e86686fbf702 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:14.290805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d357637-4d9b-4f9f-acd8-e86686fbf702 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:14.292248 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d357637-4d9b-4f9f-acd8-e86686fbf702 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-5d357637-4d9b-4f9f-acd8-e86686fbf702 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:03:14.315593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:14.316545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:14.317501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:14.317501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:14.317501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:03:14.317501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:03:14.317501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:03:14.317501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:03:14.317501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:03:14.317501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:03:14.317501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 15 10:03:14.317501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:14.317501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 423/863] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:03:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:14.322976 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccfbf92-68ac-46cf-bfb5-6ebfef21c76c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:14.424227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:14.432924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccfbf92-68ac-46cf-bfb5-6ebfef21c76c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:14.468794 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382b8c6-3ca9-4e0d-91bb-6d319fc788af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:14.469511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382b8c6-3ca9-4e0d-91bb-6d319fc788af None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:14.469800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382b8c6-3ca9-4e0d-91bb-6d319fc788af None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:14.470082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382b8c6-3ca9-4e0d-91bb-6d319fc788af None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:14.470944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382b8c6-3ca9-4e0d-91bb-6d319fc788af None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:14.471259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382b8c6-3ca9-4e0d-91bb-6d319fc788af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:14.472940 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382b8c6-3ca9-4e0d-91bb-6d319fc788af None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:14.493129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 441/864] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Jun 15 10:03:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:14.502087 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d357637-4d9b-4f9f-acd8-e86686fbf702 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:14.564144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:14.612578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d357637-4d9b-4f9f-acd8-e86686fbf702 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:14.648607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f5460-cddf-4b8d-9cfd-3d9c132d74f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:14.649345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f5460-cddf-4b8d-9cfd-3d9c132d74f1 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:14.649634 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f5460-cddf-4b8d-9cfd-3d9c132d74f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:14.649914 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f5460-cddf-4b8d-9cfd-3d9c132d74f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:14.727759 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f5460-cddf-4b8d-9cfd-3d9c132d74f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:14.729310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f5460-cddf-4b8d-9cfd-3d9c132d74f1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:14.786818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:14.838043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 424/865] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:03:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:03:14.844087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382b8c6-3ca9-4e0d-91bb-6d319fc788af None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:14.954954 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382b8c6-3ca9-4e0d-91bb-6d319fc788af None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:14.994673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e387d41-b255-4136-8d99-fdd277dc274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:14.995864 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e387d41-b255-4136-8d99-fdd277dc274a None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:14.996344 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e387d41-b255-4136-8d99-fdd277dc274a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:14.996821 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e387d41-b255-4136-8d99-fdd277dc274a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:15.008364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e387d41-b255-4136-8d99-fdd277dc274a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:15.010109 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e387d41-b255-4136-8d99-fdd277dc274a None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:15.029511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-1e387d41-b255-4136-8d99-fdd277dc274a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:15.029511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:15.029511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:15.029511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:15.029511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:15.029511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:15.029511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:15.029511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:15.029511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:15.029511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:15.029511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:15.029511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:15.029511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:15.029511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:15.029511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:15.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:15.031286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:15.031286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:15.031286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:15.031286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:15.033499 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 442/866] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:15.044984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f5460-cddf-4b8d-9cfd-3d9c132d74f1 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:15.141974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f5460-cddf-4b8d-9cfd-3d9c132d74f1 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:15.182523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a79d8-d79e-4dea-988a-b531b262eaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:15.183195 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a79d8-d79e-4dea-988a-b531b262eaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:15.183479 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a79d8-d79e-4dea-988a-b531b262eaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:15.183756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a79d8-d79e-4dea-988a-b531b262eaa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:15.185136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a79d8-d79e-4dea-988a-b531b262eaa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:15.185442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a79d8-d79e-4dea-988a-b531b262eaa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:15.186946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a79d8-d79e-4dea-988a-b531b262eaa5 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:15.199235 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 425/867] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:15.205455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e387d41-b255-4136-8d99-fdd277dc274a None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:15.210303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:15.317966 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e387d41-b255-4136-8d99-fdd277dc274a None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:15.355363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07013e0-b0f4-4dd5-90b6-09b59fabffad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:15.357421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07013e0-b0f4-4dd5-90b6-09b59fabffad None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:15.357854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07013e0-b0f4-4dd5-90b6-09b59fabffad None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:15.358361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07013e0-b0f4-4dd5-90b6-09b59fabffad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:15.366135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07013e0-b0f4-4dd5-90b6-09b59fabffad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:15.367793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07013e0-b0f4-4dd5-90b6-09b59fabffad None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:15.382979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-d07013e0-b0f4-4dd5-90b6-09b59fabffad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:15.382979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:15.382979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:15.382979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:15.382979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:15.382979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:15.382979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:15.382979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:15.382979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:15.382979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:15.382979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:15.382979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:15.382979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:03:15.382979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:03:15.382979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:15.384105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:15.385183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:03:15.385183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:03:15.385183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:03:15.385183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:03:15.385183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:03:15.385183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:03:15.385183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:15.385183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:15.386536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 443/868] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:15.393211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a79d8-d79e-4dea-988a-b531b262eaa5 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:15.414740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:15.499258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a79d8-d79e-4dea-988a-b531b262eaa5 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:15.539399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7d294-4133-48a3-862f-abd6b3086b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:15.544999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7d294-4133-48a3-862f-abd6b3086b02 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:15.544999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7d294-4133-48a3-862f-abd6b3086b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:15.544999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7d294-4133-48a3-862f-abd6b3086b02 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:15.544999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7d294-4133-48a3-862f-abd6b3086b02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:15.544999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7d294-4133-48a3-862f-abd6b3086b02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:15.544999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7d294-4133-48a3-862f-abd6b3086b02 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:15.563540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 426/869] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:03:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:15.569344 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07013e0-b0f4-4dd5-90b6-09b59fabffad None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:15.639116 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:15.679519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07013e0-b0f4-4dd5-90b6-09b59fabffad None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:15.715780 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26876ef-d094-4feb-9069-4c337363963a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:15.716979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26876ef-d094-4feb-9069-4c337363963a None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:15.717272 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26876ef-d094-4feb-9069-4c337363963a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:15.717554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26876ef-d094-4feb-9069-4c337363963a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:15.718399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26876ef-d094-4feb-9069-4c337363963a None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=589e66c6228a44adbc9e418794d7d91b) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:15.718713 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26876ef-d094-4feb-9069-4c337363963a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:15.720167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26876ef-d094-4feb-9069-4c337363963a None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:15.727257 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 444/870] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:03:15 2022] GET /identity/v3/role_assignments?user.id=589e66c6228a44adbc9e418794d7d91b&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:15.867258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db109c-a4d7-4108-b5dc-c6ed955dbecd None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:15.867586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db109c-a4d7-4108-b5dc-c6ed955dbecd None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:15.867766 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db109c-a4d7-4108-b5dc-c6ed955dbecd None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:15.869479 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 427/871] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:15.875267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfd923-5ccf-4ffb-85a8-76cb262eee57 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:15.875584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfd923-5ccf-4ffb-85a8-76cb262eee57 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:15.875839 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfd923-5ccf-4ffb-85a8-76cb262eee57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:15.894240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-e1bfd923-5ccf-4ffb-85a8-76cb262eee57 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:15.895462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e1bfd923-5ccf-4ffb-85a8-76cb262eee57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:15.978272 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 445/872] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:16.003016 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db109c-a4d7-4108-b5dc-c6ed955dbecd None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:16.019072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72a32d-21f8-4dbe-9b4e-50004c32b20d None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:16.019383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72a32d-21f8-4dbe-9b4e-50004c32b20d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:16.019642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72a32d-21f8-4dbe-9b4e-50004c32b20d None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:16.021328 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 446/873] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:03:16.026531 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d5f30-574f-4b4a-be29-e94fdf70ff47 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:16.027091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d5f30-574f-4b4a-be29-e94fdf70ff47 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:16.027578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d5f30-574f-4b4a-be29-e94fdf70ff47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:16.048481 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-878d5f30-574f-4b4a-be29-e94fdf70ff47 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:16.049870 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-878d5f30-574f-4b4a-be29-e94fdf70ff47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:16.098089 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:16.126877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:16.128092 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db109c-a4d7-4108-b5dc-c6ed955dbecd None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:16.149063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 447/874] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:16.175798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d5f30-574f-4b4a-be29-e94fdf70ff47 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:16.181049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12531e-0642-43c0-ae8c-c11c7740d8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:16.182103 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12531e-0642-43c0-ae8c-c11c7740d8f0 None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:16.182589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12531e-0642-43c0-ae8c-c11c7740d8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:16.183112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12531e-0642-43c0-ae8c-c11c7740d8f0 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:16.184151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12531e-0642-43c0-ae8c-c11c7740d8f0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:16.186626 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12531e-0642-43c0-ae8c-c11c7740d8f0 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:16.268117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 428/875] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jun 15 10:03:16 2022] POST /identity/v3/services => generated 251 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:03:16.289208 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d5f30-574f-4b4a-be29-e94fdf70ff47 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:16.347048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1832609-0d79-45bf-8dea-93486e0bcefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:16.348316 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1832609-0d79-45bf-8dea-93486e0bcefe None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:16.348902 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1832609-0d79-45bf-8dea-93486e0bcefe None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:16.349458 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1832609-0d79-45bf-8dea-93486e0bcefe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:16.363831 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1832609-0d79-45bf-8dea-93486e0bcefe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-f1832609-0d79-45bf-8dea-93486e0bcefe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:03:16.373857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:16.374874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:16.375730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:16.375730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:16.375730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:16.375730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:16.375730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:03:16.375730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:03:16.375730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:03:16.375730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:03:16.375730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:03:16.375730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:03:16.375730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:03:16.375730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:16.379090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 448/876] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Jun 15 10:03:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:16.385341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12531e-0642-43c0-ae8c-c11c7740d8f0 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:16.415746 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c8c18-47d8-437f-84b3-f11907202134 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:16.416083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c8c18-47d8-437f-84b3-f11907202134 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:16.416318 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c8c18-47d8-437f-84b3-f11907202134 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:16.419300 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 449/877] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:16.423834 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6a14b-acd4-4784-9680-9c19cd013355 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:16.424117 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6a14b-acd4-4784-9680-9c19cd013355 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:16.424395 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6a14b-acd4-4784-9680-9c19cd013355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:16.446709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-17b6a14b-acd4-4784-9680-9c19cd013355 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:16.448264 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-17b6a14b-acd4-4784-9680-9c19cd013355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:16.496532 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:16.516173 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12531e-0642-43c0-ae8c-c11c7740d8f0 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:16.549190 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 450/878] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:16 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:16.562656 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e30d6b-11ed-4be7-a23b-30ed72446764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:16.563345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e30d6b-11ed-4be7-a23b-30ed72446764 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:16.563629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e30d6b-11ed-4be7-a23b-30ed72446764 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:16.563911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e30d6b-11ed-4be7-a23b-30ed72446764 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:16.564682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e30d6b-11ed-4be7-a23b-30ed72446764 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:16.565030 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e30d6b-11ed-4be7-a23b-30ed72446764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:16.566408 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e30d6b-11ed-4be7-a23b-30ed72446764 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:16.567945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:16.569169 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6a14b-acd4-4784-9680-9c19cd013355 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:16.578060 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 429/879] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Jun 15 10:03:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:16.583422 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e30d6b-11ed-4be7-a23b-30ed72446764 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:16.694375 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6a14b-acd4-4784-9680-9c19cd013355 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:16.709232 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e30d6b-11ed-4be7-a23b-30ed72446764 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:16.732487 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b40e2-f6f6-4b70-89f4-8d32f13c98bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:16.733262 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b40e2-f6f6-4b70-89f4-8d32f13c98bb None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:16.733545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b40e2-f6f6-4b70-89f4-8d32f13c98bb None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:16.733831 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b40e2-f6f6-4b70-89f4-8d32f13c98bb None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:16.734547 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b40e2-f6f6-4b70-89f4-8d32f13c98bb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:16.735946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b40e2-f6f6-4b70-89f4-8d32f13c98bb None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:16.749554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7e94e-75a2-45e2-aa7d-eb70583816e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:16.750641 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7e94e-75a2-45e2-aa7d-eb70583816e8 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:16.751085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7e94e-75a2-45e2-aa7d-eb70583816e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:16.753074 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7e94e-75a2-45e2-aa7d-eb70583816e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:16.762422 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7e94e-75a2-45e2-aa7d-eb70583816e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:16.764143 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7e94e-75a2-45e2-aa7d-eb70583816e8 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:16.782905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-d7f7e94e-75a2-45e2-aa7d-eb70583816e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:16.782905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:16.782905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:16.782905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:16.782905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:16.782905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:16.782905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:16.782905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:16.782905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:16.782905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:16.782905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:16.782905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:16.782905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:16.782905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:16.782905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:16.783826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:16.784712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:16.784712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:16.784712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:16.784712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:16.786963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 430/880] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:16.792336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7e94e-75a2-45e2-aa7d-eb70583816e8 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:16.842011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 451/881] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jun 15 10:03:16 2022] POST /identity/v3/services => generated 269 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:03:16.853029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:16.924612 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7e94e-75a2-45e2-aa7d-eb70583816e8 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:16.973246 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac37eb-87a8-45ad-99bc-4ed9325c61b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:16.974001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac37eb-87a8-45ad-99bc-4ed9325c61b8 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:16.974263 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac37eb-87a8-45ad-99bc-4ed9325c61b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:16.974536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac37eb-87a8-45ad-99bc-4ed9325c61b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:16.975316 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac37eb-87a8-45ad-99bc-4ed9325c61b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:16.975620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac37eb-87a8-45ad-99bc-4ed9325c61b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:16.977041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac37eb-87a8-45ad-99bc-4ed9325c61b8 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:16.991232 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 431/882] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:16.996877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b40e2-f6f6-4b70-89f4-8d32f13c98bb None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:17.037133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:17.107316 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b40e2-f6f6-4b70-89f4-8d32f13c98bb None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:17.143677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526dccdc-3487-4c54-ac05-acaea8c85d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:17.144495 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526dccdc-3487-4c54-ac05-acaea8c85d5c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:17.144775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526dccdc-3487-4c54-ac05-acaea8c85d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:17.145076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526dccdc-3487-4c54-ac05-acaea8c85d5c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:17.153169 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526dccdc-3487-4c54-ac05-acaea8c85d5c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:17.157155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526dccdc-3487-4c54-ac05-acaea8c85d5c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-526dccdc-3487-4c54-ac05-acaea8c85d5c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:03:17.170803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:17.171852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:17.173189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:17.173189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:17.173189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:03:17.173189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:03:17.173189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:03:17.173189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:03:17.173189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:03:17.173189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:03:17.173189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:03:17.173189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:17.173189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 452/883] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jun 15 10:03:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:17.178524 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac37eb-87a8-45ad-99bc-4ed9325c61b8 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:17.284748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac37eb-87a8-45ad-99bc-4ed9325c61b8 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:17.291836 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:17.320056 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62974009-01ce-43cb-8a43-348b60ef3834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:17.320739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62974009-01ce-43cb-8a43-348b60ef3834 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:17.321076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62974009-01ce-43cb-8a43-348b60ef3834 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:17.321380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62974009-01ce-43cb-8a43-348b60ef3834 None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:17.322231 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62974009-01ce-43cb-8a43-348b60ef3834 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:17.322586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62974009-01ce-43cb-8a43-348b60ef3834 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:17.324020 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62974009-01ce-43cb-8a43-348b60ef3834 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:17.342582 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 432/884] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Jun 15 10:03:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:17.348180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526dccdc-3487-4c54-ac05-acaea8c85d5c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:17.447928 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:17.456043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526dccdc-3487-4c54-ac05-acaea8c85d5c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:17.491624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78db5dc-24e1-4b69-8847-ab645abffdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:17.492285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78db5dc-24e1-4b69-8847-ab645abffdbc None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:17.492570 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78db5dc-24e1-4b69-8847-ab645abffdbc None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:17.492848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78db5dc-24e1-4b69-8847-ab645abffdbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:17.576671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78db5dc-24e1-4b69-8847-ab645abffdbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:17.578172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78db5dc-24e1-4b69-8847-ab645abffdbc None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:17.676794 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:17.684667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 453/885] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:03:17 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:03:17.690080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62974009-01ce-43cb-8a43-348b60ef3834 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:17.804815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62974009-01ce-43cb-8a43-348b60ef3834 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:17.842531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741511f8-3a2c-40e9-8617-6f74d053111c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:17.843341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741511f8-3a2c-40e9-8617-6f74d053111c None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:17.843624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741511f8-3a2c-40e9-8617-6f74d053111c None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:17.843905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741511f8-3a2c-40e9-8617-6f74d053111c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:17.854439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741511f8-3a2c-40e9-8617-6f74d053111c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:17.855898 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741511f8-3a2c-40e9-8617-6f74d053111c None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:17.875006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-741511f8-3a2c-40e9-8617-6f74d053111c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:17.875006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:17.875006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:17.875006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:17.875006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:17.875006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:17.875006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:17.875006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:17.875006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:17.875006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:17.875006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:17.875006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:17.875006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:17.875006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:17.875006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:17.875932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:17.876814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:17.876814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:17.876814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:17.876814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:17.876814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 433/886] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:17.886066 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78db5dc-24e1-4b69-8847-ab645abffdbc None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:17.998742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78db5dc-24e1-4b69-8847-ab645abffdbc None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:18.006408 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9cfd8-4d23-4621-a441-72ef7acf03d2 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:18.006798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9cfd8-4d23-4621-a441-72ef7acf03d2 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:18.007151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9cfd8-4d23-4621-a441-72ef7acf03d2 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:18.008897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 434/887] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:18.014164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98494d2b-6629-4112-bf04-c9beda6ae717 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:18.014525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98494d2b-6629-4112-bf04-c9beda6ae717 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:18.014885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98494d2b-6629-4112-bf04-c9beda6ae717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:18.032637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-98494d2b-6629-4112-bf04-c9beda6ae717 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:18.034097 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-98494d2b-6629-4112-bf04-c9beda6ae717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:18.036424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b697ca7-8329-499e-9715-cdfed4d7a7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:18.037320 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b697ca7-8329-499e-9715-cdfed4d7a7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:18.037603 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b697ca7-8329-499e-9715-cdfed4d7a7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:18.037878 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b697ca7-8329-499e-9715-cdfed4d7a7b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:18.038607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b697ca7-8329-499e-9715-cdfed4d7a7b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:18.038910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b697ca7-8329-499e-9715-cdfed4d7a7b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:18.040281 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b697ca7-8329-499e-9715-cdfed4d7a7b0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:18.053486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 454/888] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:18.054537 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:18.059220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b697ca7-8329-499e-9715-cdfed4d7a7b0 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:18.117289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 435/889] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:18.128431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:18.140101 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98494d2b-6629-4112-bf04-c9beda6ae717 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:18.187005 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b697ca7-8329-499e-9715-cdfed4d7a7b0 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:18.225519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091b92b-ce82-41eb-842c-baed446b8e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:18.226338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091b92b-ce82-41eb-842c-baed446b8e2a None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:18.226625 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091b92b-ce82-41eb-842c-baed446b8e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:18.226945 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091b92b-ce82-41eb-842c-baed446b8e2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:18.235519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091b92b-ce82-41eb-842c-baed446b8e2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:18.237041 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091b92b-ce82-41eb-842c-baed446b8e2a None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:18.253886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-5091b92b-ce82-41eb-842c-baed446b8e2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:18.253886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:18.253886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:18.253886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:18.253886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:18.253886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:18.253886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:18.253886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:18.253886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:18.253886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:18.253886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:18.253886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:18.253886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:03:18.253886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:03:18.253886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:18.254855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:18.255791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:03:18.255791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:03:18.255791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:03:18.255791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:03:18.255791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:03:18.255791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:03:18.255791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:18.255791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:18.260176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 455/890] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:18.261155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091b92b-ce82-41eb-842c-baed446b8e2a None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:18.277607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98494d2b-6629-4112-bf04-c9beda6ae717 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:18.319172 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173bcf8-1354-42bf-a2ad-a7d04673e146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:18.320352 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173bcf8-1354-42bf-a2ad-a7d04673e146 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:18.320852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173bcf8-1354-42bf-a2ad-a7d04673e146 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:18.321373 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173bcf8-1354-42bf-a2ad-a7d04673e146 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:18.322266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173bcf8-1354-42bf-a2ad-a7d04673e146 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:18.323846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173bcf8-1354-42bf-a2ad-a7d04673e146 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-f173bcf8-1354-42bf-a2ad-a7d04673e146 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:18.337885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:18.338940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:18.339782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:18.339782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:18.339782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:18.339782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:18.339782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:18.339782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:18.339782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:18.339782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:18.339782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:18.339782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:18.339782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:18.339782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:18.339782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:03:18.339782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:18.347828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 436/891] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jun 15 10:03:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:18.349031 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173bcf8-1354-42bf-a2ad-a7d04673e146 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:18.361506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:18.372040 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091b92b-ce82-41eb-842c-baed446b8e2a None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:18.387843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:18.410190 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034da35-ec17-4fc2-9b99-3a8c4d5312c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:18.414131 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034da35-ec17-4fc2-9b99-3a8c4d5312c4 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:18.414131 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034da35-ec17-4fc2-9b99-3a8c4d5312c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:18.414131 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034da35-ec17-4fc2-9b99-3a8c4d5312c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:18.414131 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034da35-ec17-4fc2-9b99-3a8c4d5312c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:18.414131 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034da35-ec17-4fc2-9b99-3a8c4d5312c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:18.415370 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034da35-ec17-4fc2-9b99-3a8c4d5312c4 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:18.434534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 456/892] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:03:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:18.436627 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034da35-ec17-4fc2-9b99-3a8c4d5312c4 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:18.464610 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173bcf8-1354-42bf-a2ad-a7d04673e146 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:18.502541 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205c2d5-5cc2-4fad-8dea-70146824b84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:18.503640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205c2d5-5cc2-4fad-8dea-70146824b84f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:18.504139 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205c2d5-5cc2-4fad-8dea-70146824b84f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:18.504622 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205c2d5-5cc2-4fad-8dea-70146824b84f None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:18.505706 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205c2d5-5cc2-4fad-8dea-70146824b84f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:18.506291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205c2d5-5cc2-4fad-8dea-70146824b84f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:18.508007 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205c2d5-5cc2-4fad-8dea-70146824b84f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:18.518645 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 437/893] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jun 15 10:03:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:18.519851 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205c2d5-5cc2-4fad-8dea-70146824b84f None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:18.553459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034da35-ec17-4fc2-9b99-3a8c4d5312c4 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:18.595530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cc90b-02d9-46b7-a3fd-409f2a5fafe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:18.596124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cc90b-02d9-46b7-a3fd-409f2a5fafe9 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:18.596412 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cc90b-02d9-46b7-a3fd-409f2a5fafe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:18.596693 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cc90b-02d9-46b7-a3fd-409f2a5fafe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:18.597731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cc90b-02d9-46b7-a3fd-409f2a5fafe9 None admin] RBAC: Adding query filter params (role.id=b1c676dcdeaa4ac0a2a3a7ef96317fa0, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=3b37fadf589c40d28e8df46554d51d01) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:18.598055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cc90b-02d9-46b7-a3fd-409f2a5fafe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:18.599671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cc90b-02d9-46b7-a3fd-409f2a5fafe9 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:18.611576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 457/894] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:03:18 2022] GET /identity/v3/role_assignments?user.id=3b37fadf589c40d28e8df46554d51d01&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=b1c676dcdeaa4ac0a2a3a7ef96317fa0 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:18.613270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:18.626976 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44561c4-8135-4bc3-8e65-7725ab190bcd None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:18.627267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44561c4-8135-4bc3-8e65-7725ab190bcd None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:18.627550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44561c4-8135-4bc3-8e65-7725ab190bcd None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:18.633326 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 458/895] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:03:18.635272 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5d0c8-fbd0-4eec-b734-13675022537a None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:18.635632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5d0c8-fbd0-4eec-b734-13675022537a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:18.635989 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5d0c8-fbd0-4eec-b734-13675022537a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:18.644864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205c2d5-5cc2-4fad-8dea-70146824b84f None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:18.654900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-eab5d0c8-fbd0-4eec-b734-13675022537a None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:18.656213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-eab5d0c8-fbd0-4eec-b734-13675022537a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:18.686017 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805d1dc-e217-42b1-a231-ec4adf07b8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:18.686816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805d1dc-e217-42b1-a231-ec4adf07b8bf None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:18.687107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805d1dc-e217-42b1-a231-ec4adf07b8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:18.687381 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805d1dc-e217-42b1-a231-ec4adf07b8bf None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:18.688133 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805d1dc-e217-42b1-a231-ec4adf07b8bf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:18.688434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805d1dc-e217-42b1-a231-ec4adf07b8bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:18.690228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805d1dc-e217-42b1-a231-ec4adf07b8bf None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:18.691770 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:18.701911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 438/896] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jun 15 10:03:18 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:18.745192 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 459/897] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:18.769815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805d1dc-e217-42b1-a231-ec4adf07b8bf None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:18.889167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805d1dc-e217-42b1-a231-ec4adf07b8bf None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:18.926173 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5094c-6189-48aa-a5df-83d3445e1da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:18.926987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5094c-6189-48aa-a5df-83d3445e1da9 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:18.927269 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5094c-6189-48aa-a5df-83d3445e1da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:18.927548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5094c-6189-48aa-a5df-83d3445e1da9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:18.928215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5094c-6189-48aa-a5df-83d3445e1da9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:18.929616 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5094c-6189-48aa-a5df-83d3445e1da9 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:18.932412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-46b5094c-6189-48aa-a5df-83d3445e1da9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:18.943318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:18.944353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:18.945342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:18.945342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:18.945342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:18.945342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:18.945342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:18.945342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:18.945342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:18.945342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:18.945342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:18.945342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:18.945342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:18.945342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:18.945342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:03:18.945342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:18.945342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 439/898] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:03:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:18.952694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5d0c8-fbd0-4eec-b734-13675022537a None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:19.061670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5d0c8-fbd0-4eec-b734-13675022537a None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:19.102575 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d640c02-a227-4613-950e-9d4652a8e32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:19.103376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d640c02-a227-4613-950e-9d4652a8e32e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:19.103788 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d640c02-a227-4613-950e-9d4652a8e32e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:19.104072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d640c02-a227-4613-950e-9d4652a8e32e None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:19.104802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d640c02-a227-4613-950e-9d4652a8e32e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:19.105159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d640c02-a227-4613-950e-9d4652a8e32e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:19.107147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d640c02-a227-4613-950e-9d4652a8e32e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:19.112705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 460/899] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:03:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:19.118404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5094c-6189-48aa-a5df-83d3445e1da9 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:19.233361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16817eb4-3db5-49f7-8e2e-dce27c29b8b5 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:19.233647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16817eb4-3db5-49f7-8e2e-dce27c29b8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:19.234187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16817eb4-3db5-49f7-8e2e-dce27c29b8b5 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:19.235536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5094c-6189-48aa-a5df-83d3445e1da9 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:19.236143 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 461/900] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:03:19.241709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c5a84-6c7a-4dda-ad8d-860cc6263c66 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:19.242285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c5a84-6c7a-4dda-ad8d-860cc6263c66 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:19.242780 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c5a84-6c7a-4dda-ad8d-860cc6263c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:19.263179 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-bb1c5a84-6c7a-4dda-ad8d-860cc6263c66 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:19.265756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-bb1c5a84-6c7a-4dda-ad8d-860cc6263c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:19.275196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e6b46-f0d3-4ec3-87b8-cc1d000b962e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:19.276118 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e6b46-f0d3-4ec3-87b8-cc1d000b962e None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:19.276564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e6b46-f0d3-4ec3-87b8-cc1d000b962e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:19.277019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e6b46-f0d3-4ec3-87b8-cc1d000b962e None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:19.277941 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e6b46-f0d3-4ec3-87b8-cc1d000b962e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:19.278774 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e6b46-f0d3-4ec3-87b8-cc1d000b962e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:19.280309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e6b46-f0d3-4ec3-87b8-cc1d000b962e None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:19.286020 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 440/901] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:03:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:19.291963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e6b46-f0d3-4ec3-87b8-cc1d000b962e None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:19.298274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:19.357150 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 462/902] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:19.375612 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:19.380032 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c5a84-6c7a-4dda-ad8d-860cc6263c66 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:19.422076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e6b46-f0d3-4ec3-87b8-cc1d000b962e None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:19.459586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66161b7-40a8-42ec-b107-32c9a285d593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:19.460622 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66161b7-40a8-42ec-b107-32c9a285d593 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:19.461140 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66161b7-40a8-42ec-b107-32c9a285d593 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:19.461632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66161b7-40a8-42ec-b107-32c9a285d593 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:19.462645 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66161b7-40a8-42ec-b107-32c9a285d593 None admin] RBAC: Adding query filter params (interface=public, service_id=006bf7bf2f5a4820a250121cebead43a, region_id=RegionOne) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:19.463178 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66161b7-40a8-42ec-b107-32c9a285d593 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:19.464849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66161b7-40a8-42ec-b107-32c9a285d593 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:19.471659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 441/903] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jun 15 10:03:19 2022] GET /identity/v3/endpoints?service_id=006bf7bf2f5a4820a250121cebead43a&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:19.476815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66161b7-40a8-42ec-b107-32c9a285d593 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:19.492733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c5a84-6c7a-4dda-ad8d-860cc6263c66 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:19.541255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3399ef-a44e-4da7-b049-0854c48134a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:19.542075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3399ef-a44e-4da7-b049-0854c48134a3 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:19.542375 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3399ef-a44e-4da7-b049-0854c48134a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:19.542649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3399ef-a44e-4da7-b049-0854c48134a3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:19.543306 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3399ef-a44e-4da7-b049-0854c48134a3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:19.545442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3399ef-a44e-4da7-b049-0854c48134a3 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-8e3399ef-a44e-4da7-b049-0854c48134a3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:19.559709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:19.561505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:19.562593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 463/904] 10.222.0.60 () {56 vars in 1132 bytes} [Wed Jun 15 10:03:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:19.566610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3399ef-a44e-4da7-b049-0854c48134a3 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:19.596664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:19.621176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66161b7-40a8-42ec-b107-32c9a285d593 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:19.654501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:19.662666 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8ec0e-7214-4fdf-8ab4-1a1b3d1fd747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:19.663407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8ec0e-7214-4fdf-8ab4-1a1b3d1fd747 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:19.667561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8ec0e-7214-4fdf-8ab4-1a1b3d1fd747 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:19.667561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8ec0e-7214-4fdf-8ab4-1a1b3d1fd747 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:19.667561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8ec0e-7214-4fdf-8ab4-1a1b3d1fd747 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:19.667561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8ec0e-7214-4fdf-8ab4-1a1b3d1fd747 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:19.680989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 442/905] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jun 15 10:03:19 2022] GET /identity/v3/services => generated 1268 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:03:19.704282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3399ef-a44e-4da7-b049-0854c48134a3 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:19.743534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04cb62-8caf-4652-bf28-6ca0ac22da8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:19.744139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04cb62-8caf-4652-bf28-6ca0ac22da8c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:19.744423 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04cb62-8caf-4652-bf28-6ca0ac22da8c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:19.744703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04cb62-8caf-4652-bf28-6ca0ac22da8c None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:19.745615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04cb62-8caf-4652-bf28-6ca0ac22da8c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:19.745937 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04cb62-8caf-4652-bf28-6ca0ac22da8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:19.747980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04cb62-8caf-4652-bf28-6ca0ac22da8c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:19.754182 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 464/906] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jun 15 10:03:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:19.762167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8ec0e-7214-4fdf-8ab4-1a1b3d1fd747 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:19.884691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8ec0e-7214-4fdf-8ab4-1a1b3d1fd747 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:19.924506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79e5f5-d237-4976-bc88-749cff6310d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:19.925464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79e5f5-d237-4976-bc88-749cff6310d4 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:19.925748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79e5f5-d237-4976-bc88-749cff6310d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:19.926020 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79e5f5-d237-4976-bc88-749cff6310d4 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:19.926741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79e5f5-d237-4976-bc88-749cff6310d4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:19.927046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79e5f5-d237-4976-bc88-749cff6310d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:19.928414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79e5f5-d237-4976-bc88-749cff6310d4 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:19.934197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 443/907] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jun 15 10:03:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:19.940157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04cb62-8caf-4652-bf28-6ca0ac22da8c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:20.033650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:20.043473 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04cb62-8caf-4652-bf28-6ca0ac22da8c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:20.080054 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c1e4a-c6e4-4e40-8aaf-14d5ec09dd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:20.080723 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c1e4a-c6e4-4e40-8aaf-14d5ec09dd51 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:20.081023 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c1e4a-c6e4-4e40-8aaf-14d5ec09dd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:20.081346 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c1e4a-c6e4-4e40-8aaf-14d5ec09dd51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:20.082141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c1e4a-c6e4-4e40-8aaf-14d5ec09dd51 None admin] RBAC: Adding query filter params (interface=public, service_id=716e9afff89d4c4983a3d1f76e72c4cd, region_id=RegionOne) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:20.082449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c1e4a-c6e4-4e40-8aaf-14d5ec09dd51 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:20.083812 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c1e4a-c6e4-4e40-8aaf-14d5ec09dd51 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:20.089294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 465/908] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jun 15 10:03:19 2022] GET /identity/v3/endpoints?service_id=716e9afff89d4c4983a3d1f76e72c4cd&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:20.094447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79e5f5-d237-4976-bc88-749cff6310d4 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:20.154761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:20.202189 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79e5f5-d237-4976-bc88-749cff6310d4 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:20.238123 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca169a38-122b-4abe-9e51-9b710d2522bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:20.239198 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca169a38-122b-4abe-9e51-9b710d2522bb None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:20.239687 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca169a38-122b-4abe-9e51-9b710d2522bb None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:20.240164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca169a38-122b-4abe-9e51-9b710d2522bb None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:20.241087 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca169a38-122b-4abe-9e51-9b710d2522bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:20.242769 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca169a38-122b-4abe-9e51-9b710d2522bb None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:20.249519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 444/909] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jun 15 10:03:20 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:03:20.898760 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c473c-7990-4203-be33-e4e6ae082460 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:20.899281 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c473c-7990-4203-be33-e4e6ae082460 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:20.899281 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c473c-7990-4203-be33-e4e6ae082460 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:20.900977 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 466/910] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:20.907277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06169362-2d54-4fd1-94ed-3b225c72288f None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:20.907795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06169362-2d54-4fd1-94ed-3b225c72288f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:20.907899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06169362-2d54-4fd1-94ed-3b225c72288f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:20.927095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-06169362-2d54-4fd1-94ed-3b225c72288f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:20.928338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-06169362-2d54-4fd1-94ed-3b225c72288f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:20.998592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e48b6-fad5-4bc3-b118-0700b16e2423 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:20.998918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e48b6-fad5-4bc3-b118-0700b16e2423 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:20.999142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e48b6-fad5-4bc3-b118-0700b16e2423 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:21.000773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 467/911] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:21.006049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532647bd-a3b1-49d0-8f17-3cc3741be983 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:21.006334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532647bd-a3b1-49d0-8f17-3cc3741be983 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:21.006612 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532647bd-a3b1-49d0-8f17-3cc3741be983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:21.013684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 445/912] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:21.023423 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-532647bd-a3b1-49d0-8f17-3cc3741be983 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:21.024604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-532647bd-a3b1-49d0-8f17-3cc3741be983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:21.037668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06169362-2d54-4fd1-94ed-3b225c72288f None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:21.093499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:21.105235 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 468/913] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:21.123501 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532647bd-a3b1-49d0-8f17-3cc3741be983 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:21.147613 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06169362-2d54-4fd1-94ed-3b225c72288f None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:21.186218 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e0fb9-5416-4845-8058-59eb4acde56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:21.186953 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e0fb9-5416-4845-8058-59eb4acde56c None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:21.187241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e0fb9-5416-4845-8058-59eb4acde56c None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:21.187651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e0fb9-5416-4845-8058-59eb4acde56c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:21.199147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e0fb9-5416-4845-8058-59eb4acde56c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-542e0fb9-5416-4845-8058-59eb4acde56c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:03:21.207886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:21.209014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:21.209863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:21.209863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:21.209863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:21.209863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:21.209863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:03:21.209863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:03:21.209863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:03:21.209863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:03:21.209863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:03:21.209863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:03:21.209863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:03:21.209863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:21.216359 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 446/914] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Jun 15 10:03:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:21.217951 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e0fb9-5416-4845-8058-59eb4acde56c None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:21.227079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:21.248722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532647bd-a3b1-49d0-8f17-3cc3741be983 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:21.297834 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d6efc-0eb5-49fd-b259-d24f860cb314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:21.298515 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d6efc-0eb5-49fd-b259-d24f860cb314 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:21.298805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d6efc-0eb5-49fd-b259-d24f860cb314 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:21.299081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d6efc-0eb5-49fd-b259-d24f860cb314 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:21.299825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d6efc-0eb5-49fd-b259-d24f860cb314 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:21.301801 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d6efc-0eb5-49fd-b259-d24f860cb314 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:21.338253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e0fb9-5416-4845-8058-59eb4acde56c None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:21.349888 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:21.379368 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373138db-43f7-4013-810e-8e7b87bcc916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:21.380048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373138db-43f7-4013-810e-8e7b87bcc916 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:21.380326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373138db-43f7-4013-810e-8e7b87bcc916 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:21.380597 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373138db-43f7-4013-810e-8e7b87bcc916 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:21.385009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373138db-43f7-4013-810e-8e7b87bcc916 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:21.385009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373138db-43f7-4013-810e-8e7b87bcc916 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:21.385009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373138db-43f7-4013-810e-8e7b87bcc916 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:21.385526 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 469/915] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jun 15 10:03:21 2022] POST /identity/v3/services => generated 241 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:03:21.397317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 447/916] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Jun 15 10:03:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:21.404480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d6efc-0eb5-49fd-b259-d24f860cb314 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:21.535076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d6efc-0eb5-49fd-b259-d24f860cb314 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:21.574362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db53d3-daea-44b3-862c-fb96523c9d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:21.575684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db53d3-daea-44b3-862c-fb96523c9d6e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:21.576028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db53d3-daea-44b3-862c-fb96523c9d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:21.576340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db53d3-daea-44b3-862c-fb96523c9d6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:21.586564 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:21.588913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db53d3-daea-44b3-862c-fb96523c9d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:21.591514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db53d3-daea-44b3-862c-fb96523c9d6e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:21.609975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-00db53d3-daea-44b3-862c-fb96523c9d6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:21.609975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:21.609975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:21.609975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:21.609975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:21.609975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:21.609975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:21.609975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:21.609975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:21.609975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:21.609975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:21.609975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:21.609975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:21.609975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:21.609975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:21.611212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:21.612126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:21.612126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:21.612126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:21.612126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:21.612126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 470/917] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:21.617300 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373138db-43f7-4013-810e-8e7b87bcc916 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:21.726932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373138db-43f7-4013-810e-8e7b87bcc916 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:21.762258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bb858-d553-4ef5-bffc-23885c400f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:21.763083 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bb858-d553-4ef5-bffc-23885c400f8b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:21.763445 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bb858-d553-4ef5-bffc-23885c400f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:21.763721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bb858-d553-4ef5-bffc-23885c400f8b None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:21.764461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93bb858-d553-4ef5-bffc-23885c400f8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:21.764764 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93bb858-d553-4ef5-bffc-23885c400f8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:21.766287 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93bb858-d553-4ef5-bffc-23885c400f8b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:21.779620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 448/918] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:21.785575 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db53d3-daea-44b3-862c-fb96523c9d6e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:21.898441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db53d3-daea-44b3-862c-fb96523c9d6e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:21.935997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b79a1e-af81-4832-a782-d9f317f0e51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:21.937678 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b79a1e-af81-4832-a782-d9f317f0e51f None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:21.938173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b79a1e-af81-4832-a782-d9f317f0e51f None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:21.938652 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b79a1e-af81-4832-a782-d9f317f0e51f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:21.947340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b79a1e-af81-4832-a782-d9f317f0e51f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:21.949093 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b79a1e-af81-4832-a782-d9f317f0e51f None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-e4b79a1e-af81-4832-a782-d9f317f0e51f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:03:21.967013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:21.968008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:21.968948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:21.968948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:21.968948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:03:21.968948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:03:21.968948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:03:21.968948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:03:21.968948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:03:21.968948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:03:21.968948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:03:21.968948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:21.971676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 471/919] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jun 15 10:03:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:21.977689 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bb858-d553-4ef5-bffc-23885c400f8b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:22.016328 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:22.058614 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5bb67-9b35-4156-8dd6-002f0cd31c7c None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:22.058805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5bb67-9b35-4156-8dd6-002f0cd31c7c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:22.059205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5bb67-9b35-4156-8dd6-002f0cd31c7c None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:22.061066 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 472/920] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:22.066707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b56b0-eac4-4847-a1a8-1dcc36629f04 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:22.067002 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b56b0-eac4-4847-a1a8-1dcc36629f04 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:22.067287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b56b0-eac4-4847-a1a8-1dcc36629f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:22.086665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-732b56b0-eac4-4847-a1a8-1dcc36629f04 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:22.087862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-732b56b0-eac4-4847-a1a8-1dcc36629f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:22.096208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bb858-d553-4ef5-bffc-23885c400f8b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:22.132050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2e82e-b307-4b26-a5a6-88b497d6b7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:22.133211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2e82e-b307-4b26-a5a6-88b497d6b7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:22.133698 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2e82e-b307-4b26-a5a6-88b497d6b7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:22.134176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2e82e-b307-4b26-a5a6-88b497d6b7a0 None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:22.135247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2e82e-b307-4b26-a5a6-88b497d6b7a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:22.135758 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2e82e-b307-4b26-a5a6-88b497d6b7a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:22.138048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2e82e-b307-4b26-a5a6-88b497d6b7a0 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:22.142565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:22.157478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 449/921] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Jun 15 10:03:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:22.163433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2e82e-b307-4b26-a5a6-88b497d6b7a0 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:22.172303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 473/922] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:22.196414 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b56b0-eac4-4847-a1a8-1dcc36629f04 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:22.270845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:22.282338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2e82e-b307-4b26-a5a6-88b497d6b7a0 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:22.305459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b56b0-eac4-4847-a1a8-1dcc36629f04 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:22.323181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a6077-fd96-49e1-9bb7-e3c6528315af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:22.324330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a6077-fd96-49e1-9bb7-e3c6528315af None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:22.325447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a6077-fd96-49e1-9bb7-e3c6528315af None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:22.326272 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a6077-fd96-49e1-9bb7-e3c6528315af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:22.337975 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a6077-fd96-49e1-9bb7-e3c6528315af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:22.339864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a6077-fd96-49e1-9bb7-e3c6528315af None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:22.344471 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980ac79-5e80-4806-815a-b3f27a4962b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:22.345358 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980ac79-5e80-4806-815a-b3f27a4962b9 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:22.345641 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980ac79-5e80-4806-815a-b3f27a4962b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:22.345915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980ac79-5e80-4806-815a-b3f27a4962b9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:22.346572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980ac79-5e80-4806-815a-b3f27a4962b9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:22.348173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980ac79-5e80-4806-815a-b3f27a4962b9 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:22.361405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-e10a6077-fd96-49e1-9bb7-e3c6528315af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:22.361405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:22.361405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:22.361405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:22.361405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:22.361405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:22.361405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:22.361405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:22.361405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:22.361405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:22.361405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:22.361405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:22.361405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:22.361405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:22.361405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:22.362369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:22.363250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:22.363250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:22.363250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:22.363250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-9980ac79-5e80-4806-815a-b3f27a4962b9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:22.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:22.364352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:22.365215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:22.365215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:22.365215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:22.365215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:22.365215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:22.365215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:22.365215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:22.365215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:22.365215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:22.365215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:22.365215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:22.365215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:22.365215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 15 10:03:22.365215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:22.368581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 474/924] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:03:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:22.374329 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980ac79-5e80-4806-815a-b3f27a4962b9 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:22.374536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 450/924] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:22.375291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a6077-fd96-49e1-9bb7-e3c6528315af None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:22.434124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:22.490988 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980ac79-5e80-4806-815a-b3f27a4962b9 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:22.498046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a6077-fd96-49e1-9bb7-e3c6528315af None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:22.531056 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4a771-193f-4e21-a05c-4e01448d73b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:22.531996 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4a771-193f-4e21-a05c-4e01448d73b3 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:22.532703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4a771-193f-4e21-a05c-4e01448d73b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:22.533021 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4a771-193f-4e21-a05c-4e01448d73b3 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:22.533918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4a771-193f-4e21-a05c-4e01448d73b3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:22.534234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4a771-193f-4e21-a05c-4e01448d73b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:22.535617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4a771-193f-4e21-a05c-4e01448d73b3 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:22.537132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70265ad7-6c00-458a-858b-7eb85c8fdfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:22.537810 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70265ad7-6c00-458a-858b-7eb85c8fdfdf None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:22.538091 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70265ad7-6c00-458a-858b-7eb85c8fdfdf None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:22.538368 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70265ad7-6c00-458a-858b-7eb85c8fdfdf None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:22.539099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70265ad7-6c00-458a-858b-7eb85c8fdfdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:22.539404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70265ad7-6c00-458a-858b-7eb85c8fdfdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:22.540764 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70265ad7-6c00-458a-858b-7eb85c8fdfdf None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:22.549451 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 475/925] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:03:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:22.549451 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a7236-99d8-4afe-ba34-35b3b9de0021 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:22.549451 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a7236-99d8-4afe-ba34-35b3b9de0021 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:22.549451 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a7236-99d8-4afe-ba34-35b3b9de0021 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:22.549451 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 476/926] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:22.550574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a7236-99d8-4afe-ba34-35b3b9de0021 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:22.563189 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 451/927] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:22.563189 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db819c7-8e04-4cd3-a9ad-e93dea8c1060 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:22.563452 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db819c7-8e04-4cd3-a9ad-e93dea8c1060 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:22.564017 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db819c7-8e04-4cd3-a9ad-e93dea8c1060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:22.571037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:22.584854 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-3db819c7-8e04-4cd3-a9ad-e93dea8c1060 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:22.586098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3db819c7-8e04-4cd3-a9ad-e93dea8c1060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:22.660409 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 452/928] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:22.668216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a7236-99d8-4afe-ba34-35b3b9de0021 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:22.670175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db819c7-8e04-4cd3-a9ad-e93dea8c1060 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:22.680330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:22.705957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7a6f8-3747-4a6c-869d-c1b6fbb048a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:22.706636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7a6f8-3747-4a6c-869d-c1b6fbb048a8 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:22.706924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7a6f8-3747-4a6c-869d-c1b6fbb048a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:22.707203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a7a6f8-3747-4a6c-869d-c1b6fbb048a8 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:22.707923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7a6f8-3747-4a6c-869d-c1b6fbb048a8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:22.708232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7a6f8-3747-4a6c-869d-c1b6fbb048a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:22.709715 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a7a6f8-3747-4a6c-869d-c1b6fbb048a8 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:22.714735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 477/929] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jun 15 10:03:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:22.715644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7a6f8-3747-4a6c-869d-c1b6fbb048a8 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:22.794287 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db819c7-8e04-4cd3-a9ad-e93dea8c1060 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:22.826095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:22.828299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a7a6f8-3747-4a6c-869d-c1b6fbb048a8 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:22.833692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce561f-21a8-4a14-9a00-25672e31f0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:22.839730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce561f-21a8-4a14-9a00-25672e31f0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:22.839730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce561f-21a8-4a14-9a00-25672e31f0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:22.839730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce561f-21a8-4a14-9a00-25672e31f0e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:22.844722 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce561f-21a8-4a14-9a00-25672e31f0e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:22.846932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce561f-21a8-4a14-9a00-25672e31f0e9 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:22.863549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099532bd-b483-496d-bd59-34b09c7786cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:22.864363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099532bd-b483-496d-bd59-34b09c7786cb None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:22.864674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099532bd-b483-496d-bd59-34b09c7786cb None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:22.864977 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099532bd-b483-496d-bd59-34b09c7786cb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:22.865569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-19ce561f-21a8-4a14-9a00-25672e31f0e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:22.865569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:22.865569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:22.865569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:22.865569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:22.865569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:22.865569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:22.865569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:22.865569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:22.865569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:22.865569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:22.865569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:22.865569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:03:22.865569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:03:22.865569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:22.866839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:22.867740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:03:22.867740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:03:22.867740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:03:22.867740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:03:22.867740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:03:22.867740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:03:22.867740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:22.867740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:22.868158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099532bd-b483-496d-bd59-34b09c7786cb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:22.868158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099532bd-b483-496d-bd59-34b09c7786cb None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:22.868286 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 453/930] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:22.868808 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce561f-21a8-4a14-9a00-25672e31f0e9 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-099532bd-b483-496d-bd59-34b09c7786cb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:22.882122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:22.883113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:22.884050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 478/931] 10.222.0.60 () {56 vars in 1132 bytes} [Wed Jun 15 10:03:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:22.884921 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099532bd-b483-496d-bd59-34b09c7786cb None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:22.929324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:22.985950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce561f-21a8-4a14-9a00-25672e31f0e9 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:22.995614 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099532bd-b483-496d-bd59-34b09c7786cb None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:23.022866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60ce57-b00b-4a65-bf50-effcb1ac18a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:23.023876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60ce57-b00b-4a65-bf50-effcb1ac18a6 None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:23.024332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60ce57-b00b-4a65-bf50-effcb1ac18a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:23.024765 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60ce57-b00b-4a65-bf50-effcb1ac18a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:23.025790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60ce57-b00b-4a65-bf50-effcb1ac18a6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:23.027420 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60ce57-b00b-4a65-bf50-effcb1ac18a6 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:23.031593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34e196-c542-4c24-ac58-8c58d968f899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:23.032270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34e196-c542-4c24-ac58-8c58d968f899 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:23.032544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34e196-c542-4c24-ac58-8c58d968f899 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:23.032819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34e196-c542-4c24-ac58-8c58d968f899 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:23.033647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34e196-c542-4c24-ac58-8c58d968f899 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:23.033951 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34e196-c542-4c24-ac58-8c58d968f899 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:23.035378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34e196-c542-4c24-ac58-8c58d968f899 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:23.049233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 479/932] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:03:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:23.056615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34e196-c542-4c24-ac58-8c58d968f899 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:23.121084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 454/933] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jun 15 10:03:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:03:23.121084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60ce57-b00b-4a65-bf50-effcb1ac18a6 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:23.187027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:23.191584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:23.192286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34e196-c542-4c24-ac58-8c58d968f899 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:23.244500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d1ff6-526c-4e0c-918e-769ebee3e65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:23.245224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d1ff6-526c-4e0c-918e-769ebee3e65e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:23.245511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d1ff6-526c-4e0c-918e-769ebee3e65e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:23.245788 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d1ff6-526c-4e0c-918e-769ebee3e65e None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:23.247088 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d1ff6-526c-4e0c-918e-769ebee3e65e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:23.247404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d1ff6-526c-4e0c-918e-769ebee3e65e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:23.248887 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d1ff6-526c-4e0c-918e-769ebee3e65e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:23.254848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 480/934] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jun 15 10:03:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:23.260609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d1ff6-526c-4e0c-918e-769ebee3e65e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:23.276170 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60ce57-b00b-4a65-bf50-effcb1ac18a6 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:23.320853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41328139-54e3-472a-9004-93cced5375a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:23.321821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41328139-54e3-472a-9004-93cced5375a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:23.322107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41328139-54e3-472a-9004-93cced5375a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:23.322389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41328139-54e3-472a-9004-93cced5375a6 None admin] PATH_INFO: `/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/3b37fadf589c40d28e8df46554d51d01/roles/b1c676dcdeaa4ac0a2a3a7ef96317fa0` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:23.324658 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41328139-54e3-472a-9004-93cced5375a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=db47f1bb607c4d1a9c346fdc80ecadfa, user_id=3b37fadf589c40d28e8df46554d51d01, role_id=b1c676dcdeaa4ac0a2a3a7ef96317fa0)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:23.326700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41328139-54e3-472a-9004-93cced5375a6 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:23.384315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d1ff6-526c-4e0c-918e-769ebee3e65e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:23.423093 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1495813-8fc6-418c-ab28-11c60533b1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:23.423748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1495813-8fc6-418c-ab28-11c60533b1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:23.424035 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1495813-8fc6-418c-ab28-11c60533b1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:23.424315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1495813-8fc6-418c-ab28-11c60533b1b1 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:23.425325 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1495813-8fc6-418c-ab28-11c60533b1b1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:23.425640 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1495813-8fc6-418c-ab28-11c60533b1b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:23.427025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1495813-8fc6-418c-ab28-11c60533b1b1 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:23.427676 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 455/935] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:03:23 2022] PUT /identity/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/3b37fadf589c40d28e8df46554d51d01/roles/b1c676dcdeaa4ac0a2a3a7ef96317fa0 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:03:23.439739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 481/936] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jun 15 10:03:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:23.445278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41328139-54e3-472a-9004-93cced5375a6 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:23.582776 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41328139-54e3-472a-9004-93cced5375a6 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:23.594317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:23.626961 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b91201-2664-467e-b66d-eede245ddefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:23.627654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b91201-2664-467e-b66d-eede245ddefc None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:23.628163 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b91201-2664-467e-b66d-eede245ddefc None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:23.628289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b91201-2664-467e-b66d-eede245ddefc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:23.629098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b91201-2664-467e-b66d-eede245ddefc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:23.630562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b91201-2664-467e-b66d-eede245ddefc None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:23.687285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee1211-fa95-415c-8031-1f4b9b8b1ef2 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:23.687536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee1211-fa95-415c-8031-1f4b9b8b1ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:23.687979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee1211-fa95-415c-8031-1f4b9b8b1ef2 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:23.689768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 482/937] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:03:23.695288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46769e-30ea-4362-82d7-a9d56f24ce42 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:23.695807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46769e-30ea-4362-82d7-a9d56f24ce42 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:23.696291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46769e-30ea-4362-82d7-a9d56f24ce42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:23.716076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-cc46769e-30ea-4362-82d7-a9d56f24ce42 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:23.717537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-cc46769e-30ea-4362-82d7-a9d56f24ce42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:23.736050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:23.743827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 456/938] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jun 15 10:03:23 2022] POST /identity/v3/endpoints => generated 347 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:03:23.817152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 483/939] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:23.836426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b91201-2664-467e-b66d-eede245ddefc None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:23.953773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b91201-2664-467e-b66d-eede245ddefc None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:23.990949 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87eebb-22fc-4590-a764-85bc13c8b786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:23.991762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87eebb-22fc-4590-a764-85bc13c8b786 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:23.992046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87eebb-22fc-4590-a764-85bc13c8b786 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:23.992323 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87eebb-22fc-4590-a764-85bc13c8b786 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:23.993003 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87eebb-22fc-4590-a764-85bc13c8b786 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:23.994446 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87eebb-22fc-4590-a764-85bc13c8b786 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-2f87eebb-22fc-4590-a764-85bc13c8b786 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:24.007999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:24.009071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:24.009930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:24.009930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:24.009930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:24.009930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:24.009930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:24.009930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:24.009930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:24.009930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:24.009930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:24.009930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:24.009930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:24.009930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:24.009930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:03:24.009930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:24.013104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 457/940] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jun 15 10:03:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:24.015764 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46769e-30ea-4362-82d7-a9d56f24ce42 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:24.123054 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46769e-30ea-4362-82d7-a9d56f24ce42 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:24.160041 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0d470-4986-4f0a-93cf-05d967d0e453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:24.160717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0d470-4986-4f0a-93cf-05d967d0e453 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:24.161018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0d470-4986-4f0a-93cf-05d967d0e453 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:24.161297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0d470-4986-4f0a-93cf-05d967d0e453 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:24.162018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0d470-4986-4f0a-93cf-05d967d0e453 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:24.162321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0d470-4986-4f0a-93cf-05d967d0e453 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:24.163698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0d470-4986-4f0a-93cf-05d967d0e453 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:24.170083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 484/941] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jun 15 10:03:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:24.175917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87eebb-22fc-4590-a764-85bc13c8b786 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:24.211982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:24.285833 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87eebb-22fc-4590-a764-85bc13c8b786 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:24.331428 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ef55f-12df-4134-9ac3-dd4c0da78600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:24.332099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ef55f-12df-4134-9ac3-dd4c0da78600 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:24.332384 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ef55f-12df-4134-9ac3-dd4c0da78600 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:24.332659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ef55f-12df-4134-9ac3-dd4c0da78600 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:24.333409 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ef55f-12df-4134-9ac3-dd4c0da78600 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:24.333720 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ef55f-12df-4134-9ac3-dd4c0da78600 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:24.335090 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ef55f-12df-4134-9ac3-dd4c0da78600 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:24.341059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 458/942] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jun 15 10:03:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:24.346739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0d470-4986-4f0a-93cf-05d967d0e453 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:24.365361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:24.461067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0d470-4986-4f0a-93cf-05d967d0e453 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:24.497826 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543bd7f2-88ba-4f16-bdb1-f94e638a834a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:24.498405 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bd7f2-88ba-4f16-bdb1-f94e638a834a None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:24.498684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bd7f2-88ba-4f16-bdb1-f94e638a834a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:24.498953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543bd7f2-88ba-4f16-bdb1-f94e638a834a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:24.499742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543bd7f2-88ba-4f16-bdb1-f94e638a834a None admin] RBAC: Adding query filter params (interface=public, service_id=b7e0b41d1cf94a8a8d31f791d2a3507a, region_id=RegionOne) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:24.500050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543bd7f2-88ba-4f16-bdb1-f94e638a834a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:24.501442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543bd7f2-88ba-4f16-bdb1-f94e638a834a None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:24.506974 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 485/943] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jun 15 10:03:24 2022] GET /identity/v3/endpoints?service_id=b7e0b41d1cf94a8a8d31f791d2a3507a&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:24.513443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ef55f-12df-4134-9ac3-dd4c0da78600 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:24.617352 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:24.629369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ef55f-12df-4134-9ac3-dd4c0da78600 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:24.666473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800f385-3511-49ae-83f3-9238af99098f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:24.667140 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800f385-3511-49ae-83f3-9238af99098f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:24.667419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800f385-3511-49ae-83f3-9238af99098f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:24.667692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800f385-3511-49ae-83f3-9238af99098f None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:24.668422 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800f385-3511-49ae-83f3-9238af99098f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:24.670479 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800f385-3511-49ae-83f3-9238af99098f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:24.677322 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 459/944] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jun 15 10:03:24 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:03:25.500536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78600b65-03be-42c4-ac39-62e9d46b2284 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:25.501601 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78600b65-03be-42c4-ac39-62e9d46b2284 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:25.502087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78600b65-03be-42c4-ac39-62e9d46b2284 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:25.504027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 486/945] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:25.509884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791484e-d509-410f-b35c-bca2286d17d7 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:25.510806 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791484e-d509-410f-b35c-bca2286d17d7 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:25.511294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791484e-d509-410f-b35c-bca2286d17d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:25.541370 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-1791484e-d509-410f-b35c-bca2286d17d7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:25.543005 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1791484e-d509-410f-b35c-bca2286d17d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:25.619232 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 460/946] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:25.644621 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78600b65-03be-42c4-ac39-62e9d46b2284 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:25.733435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:25.758176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78600b65-03be-42c4-ac39-62e9d46b2284 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:25.770065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b3cd5-070b-4275-aa5c-f64d782e1ba5 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:25.770692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b3cd5-070b-4275-aa5c-f64d782e1ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:25.771181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8b3cd5-070b-4275-aa5c-f64d782e1ba5 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:25.773056 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 461/947] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:03:25.778247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f419376-a0ee-4ec0-82c8-b65a5493cdde None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:25.778791 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f419376-a0ee-4ec0-82c8-b65a5493cdde None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:25.779272 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f419376-a0ee-4ec0-82c8-b65a5493cdde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:25.798205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-4f419376-a0ee-4ec0-82c8-b65a5493cdde None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:25.798410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb4639-c77f-425e-941f-0e1b71ac98ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:25.798879 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb4639-c77f-425e-941f-0e1b71ac98ce None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:25.799166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb4639-c77f-425e-941f-0e1b71ac98ce None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:25.799441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb4639-c77f-425e-941f-0e1b71ac98ce None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:25.800445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb4639-c77f-425e-941f-0e1b71ac98ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:25.801020 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4f419376-a0ee-4ec0-82c8-b65a5493cdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:25.801946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb4639-c77f-425e-941f-0e1b71ac98ce None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:25.803561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: WARNING keystone.server.flask.common [None req-eabb4639-c77f-425e-941f-0e1b71ac98ce None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 15 10:03:25.864233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.sql.core [None req-eabb4639-c77f-425e-941f-0e1b71ac98ce None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 10:03:25.864233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 10:03:25.864233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [parameters: {'id': '24a5b985c7564c38a5bdbafbe7f25a98', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 10:03:25.864233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80503) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 10:03:25.885044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 462/948] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:25.886214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-eabb4639-c77f-425e-941f-0e1b71ac98ce None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 24a5b985c7564c38a5bdbafbe7f25a98.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 24a5b985c7564c38a5bdbafbe7f25a98. Jun 15 10:03:25.886214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:25.886214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 10:03:25.886214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 10:03:25.886214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 10:03:25.886214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 10:03:25.886214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 10:03:25.886214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 10:03:25.886214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 10:03:25.886214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application conn.query(q) Jun 15 10:03:25.886214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 10:03:25.886214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 10:03:25.886214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application result.read() Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 15 10:03:25.887595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application next(self.gen) Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application yield resource Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application next(self.gen) Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise self.value Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application session.commit() Jun 15 10:03:25.888566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self.session.flush() Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application compat.raise_( Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 10:03:25.889855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 10:03:25.890752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 10:03:25.890752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 10:03:25.890752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 10:03:25.890752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 10:03:25.890752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 15 10:03:25.890752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 15 10:03:25.890752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 10:03:25.890752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 10:03:25.890752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 10:03:25.890752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 10:03:25.890752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 10:03:25.890752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 10:03:25.890752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 10:03:25.890752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 10:03:25.890752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application conn.query(q) Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application result.read() Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 10:03:25.891533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 10:03:25.892396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 10:03:25.892396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 10:03:25.892396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 10:03:25.892396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 10:03:25.892396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 10:03:25.892396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 10:03:25.892396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 15 10:03:25.892396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 15 10:03:25.892396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [parameters: {'id': '24a5b985c7564c38a5bdbafbe7f25a98', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 15 10:03:25.892396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 10:03:25.892396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:25.892396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:25.892396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:25.892396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:25.892396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:25.894043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:25.895664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:25.895664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 15 10:03:25.895664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 15 10:03:25.895664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:25.895664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:25.895664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 15 10:03:25.895664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 15 10:03:25.895664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 24a5b985c7564c38a5bdbafbe7f25a98. Jun 15 10:03:25.895664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:25.895664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 487/949] 10.222.0.60 () {60 vars in 1140 bytes} [Wed Jun 15 10:03:25 2022] POST /identity/v3/projects => generated 262 bytes in 245 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:25.903230 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f419376-a0ee-4ec0-82c8-b65a5493cdde None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:25.911440 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb4639-c77f-425e-941f-0e1b71ac98ce None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:25.925384 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:26.040927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f419376-a0ee-4ec0-82c8-b65a5493cdde None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:26.047275 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb4639-c77f-425e-941f-0e1b71ac98ce None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:26.079942 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:26.085793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681f6db-e252-482f-888a-2fe7ee466f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:26.086612 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681f6db-e252-482f-888a-2fe7ee466f51 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:26.086895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681f6db-e252-482f-888a-2fe7ee466f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:26.087169 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681f6db-e252-482f-888a-2fe7ee466f51 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:26.092155 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be5df6-7158-46b9-ae65-2cf1a846aec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:26.092155 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be5df6-7158-46b9-ae65-2cf1a846aec4 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:26.092155 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be5df6-7158-46b9-ae65-2cf1a846aec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:26.092155 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be5df6-7158-46b9-ae65-2cf1a846aec4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:26.097277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be5df6-7158-46b9-ae65-2cf1a846aec4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:26.098737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be5df6-7158-46b9-ae65-2cf1a846aec4 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:26.098845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681f6db-e252-482f-888a-2fe7ee466f51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-f681f6db-e252-482f-888a-2fe7ee466f51 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:03:26.107676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:26.108638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:26.109900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:26.109900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:26.109900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:26.109900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:26.109900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:03:26.109900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:03:26.109900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:03:26.109900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:03:26.109900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:03:26.109900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:03:26.109900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:03:26.109900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:26.109900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 488/950] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Jun 15 10:03:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:26.110955 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b318f3-1a83-43c6-8a7b-6425741a765d None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:26.111053 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b318f3-1a83-43c6-8a7b-6425741a765d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:26.111304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b318f3-1a83-43c6-8a7b-6425741a765d None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:26.117283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-38be5df6-7158-46b9-ae65-2cf1a846aec4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:26.117283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:26.117283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:26.117283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:26.117283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:26.117283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:26.117283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:26.117283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:26.117283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:26.117283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:26.117283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:26.117283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:26.117283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:03:26.117283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:03:26.117283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:26.118635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:26.120159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:03:26.120159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:03:26.120159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:03:26.120159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:03:26.120159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:03:26.120159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:03:26.120159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:26.120159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:26.120159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 463/951] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:26.120159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be5df6-7158-46b9-ae65-2cf1a846aec4 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:26.120712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 489/952] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:26.123420 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef4046-e2ae-4cee-b4e1-b3da6043db37 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:26.123708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef4046-e2ae-4cee-b4e1-b3da6043db37 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:26.123982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef4046-e2ae-4cee-b4e1-b3da6043db37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:26.145859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-f2ef4046-e2ae-4cee-b4e1-b3da6043db37 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:26.147425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f2ef4046-e2ae-4cee-b4e1-b3da6043db37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:26.208317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:26.225006 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 490/953] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:26.225433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef4046-e2ae-4cee-b4e1-b3da6043db37 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:26.244063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be5df6-7158-46b9-ae65-2cf1a846aec4 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:26.297580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb69ce-5591-4efc-85bb-368d381660b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:26.298195 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb69ce-5591-4efc-85bb-368d381660b2 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:26.298730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb69ce-5591-4efc-85bb-368d381660b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:26.299013 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb69ce-5591-4efc-85bb-368d381660b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:26.300472 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb69ce-5591-4efc-85bb-368d381660b2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:26.300787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb69ce-5591-4efc-85bb-368d381660b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:26.302464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb69ce-5591-4efc-85bb-368d381660b2 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:26.321118 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 464/954] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Jun 15 10:03:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:26.322357 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb69ce-5591-4efc-85bb-368d381660b2 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:26.357307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef4046-e2ae-4cee-b4e1-b3da6043db37 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:26.388627 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:26.400803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6acf0e-40be-47f8-a497-79ac1f144a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:26.401972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6acf0e-40be-47f8-a497-79ac1f144a41 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:26.403000 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6acf0e-40be-47f8-a497-79ac1f144a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:26.403512 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6acf0e-40be-47f8-a497-79ac1f144a41 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:26.404513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6acf0e-40be-47f8-a497-79ac1f144a41 None admin] RBAC: Adding query filter params (name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:26.405041 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6acf0e-40be-47f8-a497-79ac1f144a41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:26.406895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6acf0e-40be-47f8-a497-79ac1f144a41 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:26.428797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 491/955] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jun 15 10:03:26 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:26.430036 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6acf0e-40be-47f8-a497-79ac1f144a41 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:26.462692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb69ce-5591-4efc-85bb-368d381660b2 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:26.517140 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e9edc-637c-47a3-9c28-123872d036c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:26.517982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e9edc-637c-47a3-9c28-123872d036c4 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:26.519054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e9edc-637c-47a3-9c28-123872d036c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:26.519356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e9edc-637c-47a3-9c28-123872d036c4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:26.520067 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e9edc-637c-47a3-9c28-123872d036c4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:26.522354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e9edc-637c-47a3-9c28-123872d036c4 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-2d1e9edc-637c-47a3-9c28-123872d036c4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:26.538985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:26.540126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:26.541563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:26.541563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:26.541563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:26.541563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:26.541563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:26.541563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:26.541563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:26.541563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:26.541563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:26.541563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:26.541563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:26.541563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:26.541563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:03:26.541563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:26.546318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 465/956] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:26.547323 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e9edc-637c-47a3-9c28-123872d036c4 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:26.551876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:26.575698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6acf0e-40be-47f8-a497-79ac1f144a41 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:26.635122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b5cce-1c05-4e5d-905b-2de596debc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:26.635641 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b5cce-1c05-4e5d-905b-2de596debc95 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:26.635943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b5cce-1c05-4e5d-905b-2de596debc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:26.638304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b5cce-1c05-4e5d-905b-2de596debc95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:26.661511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b5cce-1c05-4e5d-905b-2de596debc95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:26.663747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b5cce-1c05-4e5d-905b-2de596debc95 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:26.691696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-2a7b5cce-1c05-4e5d-905b-2de596debc95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:26.691696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:26.691696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:26.691696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:26.691696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:26.691696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:26.691696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:26.691696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:26.691696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:26.691696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:26.691696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:26.691696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:26.691696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:26.691696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:26.691696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:26.693234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:26.694158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:26.694158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:26.694158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:26.694158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:26.694158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 492/957] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:26.700366 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b5cce-1c05-4e5d-905b-2de596debc95 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:26.722655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e9edc-637c-47a3-9c28-123872d036c4 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:26.743929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:26.772985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7df0a-0ea5-49fa-856f-e68dc9519785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:26.774329 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7df0a-0ea5-49fa-856f-e68dc9519785 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:26.774585 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7df0a-0ea5-49fa-856f-e68dc9519785 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:26.774870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7df0a-0ea5-49fa-856f-e68dc9519785 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:26.775805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7df0a-0ea5-49fa-856f-e68dc9519785 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:26.776148 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7df0a-0ea5-49fa-856f-e68dc9519785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:26.779322 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7df0a-0ea5-49fa-856f-e68dc9519785 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:26.789421 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 466/958] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jun 15 10:03:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:26.795724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7df0a-0ea5-49fa-856f-e68dc9519785 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:26.849503 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b5cce-1c05-4e5d-905b-2de596debc95 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:26.907494 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:26.908804 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6478d67-52d7-4203-9235-824c98ab6f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:26.909599 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6478d67-52d7-4203-9235-824c98ab6f3a None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:26.909881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6478d67-52d7-4203-9235-824c98ab6f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:26.910164 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6478d67-52d7-4203-9235-824c98ab6f3a None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:26.910905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6478d67-52d7-4203-9235-824c98ab6f3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:26.911217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6478d67-52d7-4203-9235-824c98ab6f3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:26.912609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6478d67-52d7-4203-9235-824c98ab6f3a None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:26.927535 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 493/959] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:26.932607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6478d67-52d7-4203-9235-824c98ab6f3a None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:26.951533 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7df0a-0ea5-49fa-856f-e68dc9519785 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:26.987675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d961ac-6c70-4fb6-acea-a77f6b66ea42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:26.988341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d961ac-6c70-4fb6-acea-a77f6b66ea42 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:26.988619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d961ac-6c70-4fb6-acea-a77f6b66ea42 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:26.988890 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d961ac-6c70-4fb6-acea-a77f6b66ea42 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:26.989629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d961ac-6c70-4fb6-acea-a77f6b66ea42 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:26.989935 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d961ac-6c70-4fb6-acea-a77f6b66ea42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:26.991353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d961ac-6c70-4fb6-acea-a77f6b66ea42 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:27.004989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 467/960] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jun 15 10:03:26 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:27.063321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:27.071131 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6478d67-52d7-4203-9235-824c98ab6f3a None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:27.119166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a296a47-a66c-4acf-a336-4554de075e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:27.119929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a296a47-a66c-4acf-a336-4554de075e4f None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:27.120212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a296a47-a66c-4acf-a336-4554de075e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:27.120491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a296a47-a66c-4acf-a336-4554de075e4f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:27.129698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a296a47-a66c-4acf-a336-4554de075e4f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:27.131176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a296a47-a66c-4acf-a336-4554de075e4f None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:27.139994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a14a17-1221-445d-bd7e-f858233e5dfd None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:27.140303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a14a17-1221-445d-bd7e-f858233e5dfd None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:27.140567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a14a17-1221-445d-bd7e-f858233e5dfd None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:27.142304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 468/961] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:27.148200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0cb9d-be06-498e-b1ee-79658c7d759f None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:27.148497 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0cb9d-be06-498e-b1ee-79658c7d759f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-7a296a47-a66c-4acf-a336-4554de075e4f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:03:27.148594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:27.149500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:27.150428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:27.150428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:27.150428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:03:27.150428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:03:27.150428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:03:27.150428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:03:27.150428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:03:27.150428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:03:27.150428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 15 10:03:27.150428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:27.150967 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0cb9d-be06-498e-b1ee-79658c7d759f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:27.151034 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 494/962] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jun 15 10:03:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:27.159228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a296a47-a66c-4acf-a336-4554de075e4f None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:27.168054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-9dd0cb9d-be06-498e-b1ee-79658c7d759f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:27.169549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9dd0cb9d-be06-498e-b1ee-79658c7d759f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:27.259126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 469/963] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:27.284517 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0cb9d-be06-498e-b1ee-79658c7d759f None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:27.296480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a296a47-a66c-4acf-a336-4554de075e4f None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:27.335659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1427e-9997-4d07-bcff-40d51152beac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:27.336729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1427e-9997-4d07-bcff-40d51152beac None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:27.337263 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1427e-9997-4d07-bcff-40d51152beac None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:27.337749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1427e-9997-4d07-bcff-40d51152beac None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:27.338854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1427e-9997-4d07-bcff-40d51152beac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:27.339366 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1427e-9997-4d07-bcff-40d51152beac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:27.341708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1427e-9997-4d07-bcff-40d51152beac None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:27.363459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 495/964] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Jun 15 10:03:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:27.364715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:27.369145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1427e-9997-4d07-bcff-40d51152beac None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:27.386573 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:27.416371 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0cb9d-be06-498e-b1ee-79658c7d759f None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:27.454531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e3ae4-0891-4f82-ab08-eabc8d018460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:27.455657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e3ae4-0891-4f82-ab08-eabc8d018460 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:27.456128 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e3ae4-0891-4f82-ab08-eabc8d018460 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:27.456588 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e3ae4-0891-4f82-ab08-eabc8d018460 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:27.457502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e3ae4-0891-4f82-ab08-eabc8d018460 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:27.459088 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e3ae4-0891-4f82-ab08-eabc8d018460 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-ca4e3ae4-0891-4f82-ab08-eabc8d018460 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:27.472102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:27.473266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:27.474285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:27.474285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:27.474285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:27.474285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:27.474285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:27.474285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:27.474285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:27.474285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:27.474285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:27.474285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:27.474285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:27.474285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:27.474285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:03:27.474285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:27.481889 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 470/965] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jun 15 10:03:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:27.482983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e3ae4-0891-4f82-ab08-eabc8d018460 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:27.493999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1427e-9997-4d07-bcff-40d51152beac None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:27.534495 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795b0e4-dbcc-443a-81d6-60071097c1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:27.535457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795b0e4-dbcc-443a-81d6-60071097c1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:27.536057 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795b0e4-dbcc-443a-81d6-60071097c1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:27.536497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795b0e4-dbcc-443a-81d6-60071097c1d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:27.619994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e3ae4-0891-4f82-ab08-eabc8d018460 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:27.630026 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795b0e4-dbcc-443a-81d6-60071097c1d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:27.632097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795b0e4-dbcc-443a-81d6-60071097c1d9 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:27.645340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:27.649035 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:27.657415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4fd05-82dd-4ce7-8ed8-20a8cd2b9abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:27.658099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4fd05-82dd-4ce7-8ed8-20a8cd2b9abc None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:27.658385 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4fd05-82dd-4ce7-8ed8-20a8cd2b9abc None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:27.665083 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4fd05-82dd-4ce7-8ed8-20a8cd2b9abc None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:27.665083 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4fd05-82dd-4ce7-8ed8-20a8cd2b9abc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:27.665083 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4fd05-82dd-4ce7-8ed8-20a8cd2b9abc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:27.665083 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f4fd05-82dd-4ce7-8ed8-20a8cd2b9abc None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:27.671235 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 471/966] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jun 15 10:03:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:27.672620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4fd05-82dd-4ce7-8ed8-20a8cd2b9abc None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:27.752396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 496/967] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:03:27 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 10:03:27.758876 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795b0e4-dbcc-443a-81d6-60071097c1d9 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:27.783427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4fd05-82dd-4ce7-8ed8-20a8cd2b9abc None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:27.819970 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1934c06-fd7c-466c-bbec-2e5b82d73833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:27.820981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1934c06-fd7c-466c-bbec-2e5b82d73833 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:27.820981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1934c06-fd7c-466c-bbec-2e5b82d73833 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:27.821236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1934c06-fd7c-466c-bbec-2e5b82d73833 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:27.821933 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1934c06-fd7c-466c-bbec-2e5b82d73833 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:27.822242 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1934c06-fd7c-466c-bbec-2e5b82d73833 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:27.823612 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1934c06-fd7c-466c-bbec-2e5b82d73833 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:27.833433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 472/968] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jun 15 10:03:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:27.834964 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1934c06-fd7c-466c-bbec-2e5b82d73833 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:27.883730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795b0e4-dbcc-443a-81d6-60071097c1d9 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:27.900062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:27.912397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:27.923360 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb8a54-55f4-4624-bd61-fc7ab503bb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:27.924504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb8a54-55f4-4624-bd61-fc7ab503bb2d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:27.925553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb8a54-55f4-4624-bd61-fc7ab503bb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:27.925938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb8a54-55f4-4624-bd61-fc7ab503bb2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:27.940726 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb8a54-55f4-4624-bd61-fc7ab503bb2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:27.942579 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb8a54-55f4-4624-bd61-fc7ab503bb2d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:27.961385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-29cb8a54-55f4-4624-bd61-fc7ab503bb2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:27.961385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:27.961385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:27.961385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:27.961385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:27.961385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:27.961385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:27.961385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:27.961385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:27.961385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:27.961385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:27.961385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:27.961385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:27.961385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:27.961385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:27.962496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:27.963596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:27.963596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:27.963596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:27.963596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:27.970662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 497/969] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:27.972638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb8a54-55f4-4624-bd61-fc7ab503bb2d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:27.992709 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1934c06-fd7c-466c-bbec-2e5b82d73833 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:28.030524 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69639bf5-80bf-47c8-9883-f6bd877a1de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:28.031189 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69639bf5-80bf-47c8-9883-f6bd877a1de0 None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:28.031471 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69639bf5-80bf-47c8-9883-f6bd877a1de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:28.031745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69639bf5-80bf-47c8-9883-f6bd877a1de0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:28.032488 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69639bf5-80bf-47c8-9883-f6bd877a1de0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:28.033882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69639bf5-80bf-47c8-9883-f6bd877a1de0 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:28.095571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb8a54-55f4-4624-bd61-fc7ab503bb2d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:28.113123 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 473/970] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jun 15 10:03:27 2022] POST /identity/v3/endpoints => generated 349 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:03:28.150391 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691a30c-3abb-4e59-aa06-0c000a36ad1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:28.151495 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691a30c-3abb-4e59-aa06-0c000a36ad1d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:28.152069 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691a30c-3abb-4e59-aa06-0c000a36ad1d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:28.153282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1691a30c-3abb-4e59-aa06-0c000a36ad1d None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:28.154444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691a30c-3abb-4e59-aa06-0c000a36ad1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:28.159848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691a30c-3abb-4e59-aa06-0c000a36ad1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:28.159848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1691a30c-3abb-4e59-aa06-0c000a36ad1d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:28.171259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 498/971] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:28.177844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69639bf5-80bf-47c8-9883-f6bd877a1de0 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:28.297273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69639bf5-80bf-47c8-9883-f6bd877a1de0 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:28.313067 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:28.339979 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a392213-a9aa-46a3-817a-7715be2d4b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:28.340847 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a392213-a9aa-46a3-817a-7715be2d4b03 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:28.341159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a392213-a9aa-46a3-817a-7715be2d4b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:28.341436 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a392213-a9aa-46a3-817a-7715be2d4b03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:28.350147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a392213-a9aa-46a3-817a-7715be2d4b03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:28.352105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a392213-a9aa-46a3-817a-7715be2d4b03 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:28.368336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-6a392213-a9aa-46a3-817a-7715be2d4b03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:28.368336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:28.368336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:28.368336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:28.368336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:28.368336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:28.368336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:28.368336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:28.368336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:28.368336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:28.368336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:28.368336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:28.368336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:03:28.368336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:03:28.368336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:28.369303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:28.370355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:03:28.370355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:03:28.370355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:03:28.370355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:03:28.370355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:03:28.370355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:03:28.370355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:28.370355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:28.370355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 474/972] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:28.377357 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691a30c-3abb-4e59-aa06-0c000a36ad1d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:28.405629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:28.497228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1691a30c-3abb-4e59-aa06-0c000a36ad1d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:28.533931 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d1fc3-1f27-4143-90b8-b79bb034383f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:28.534958 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d1fc3-1f27-4143-90b8-b79bb034383f None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:28.535450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d1fc3-1f27-4143-90b8-b79bb034383f None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:28.535930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d1fc3-1f27-4143-90b8-b79bb034383f None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:28.536929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d1fc3-1f27-4143-90b8-b79bb034383f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:28.537510 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d1fc3-1f27-4143-90b8-b79bb034383f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:28.539195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d1fc3-1f27-4143-90b8-b79bb034383f None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:28.554349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 499/973] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:03:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:28.562164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a392213-a9aa-46a3-817a-7715be2d4b03 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:28.675454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a392213-a9aa-46a3-817a-7715be2d4b03 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:28.713785 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f691cf8-38ec-4571-891b-04043c723858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:28.714799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f691cf8-38ec-4571-891b-04043c723858 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:28.715323 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f691cf8-38ec-4571-891b-04043c723858 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:28.715798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f691cf8-38ec-4571-891b-04043c723858 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:28.716864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f691cf8-38ec-4571-891b-04043c723858 None admin] RBAC: Adding query filter params (role.id=b1c676dcdeaa4ac0a2a3a7ef96317fa0, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=3b37fadf589c40d28e8df46554d51d01) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:28.717423 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f691cf8-38ec-4571-891b-04043c723858 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:28.719139 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f691cf8-38ec-4571-891b-04043c723858 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:28.726148 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 475/974] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:03:28 2022] GET /identity/v3/role_assignments?user.id=3b37fadf589c40d28e8df46554d51d01&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=b1c676dcdeaa4ac0a2a3a7ef96317fa0 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:28.845571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79a09e-781c-42d4-98cb-d01777195c0f None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:28.846122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79a09e-781c-42d4-98cb-d01777195c0f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:28.846122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79a09e-781c-42d4-98cb-d01777195c0f None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:28.847895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 500/975] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:28.853748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e965c-958b-4772-999a-4baec898e7b8 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:28.854053 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e965c-958b-4772-999a-4baec898e7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:28.854313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e965c-958b-4772-999a-4baec898e7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:28.873044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-c66e965c-958b-4772-999a-4baec898e7b8 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:28.874264 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c66e965c-958b-4772-999a-4baec898e7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:28.906466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:28.954402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 476/976] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:28.984416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79a09e-781c-42d4-98cb-d01777195c0f None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:29.075150 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:29.096676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79a09e-781c-42d4-98cb-d01777195c0f None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:29.132508 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e2bb7-9df4-4e62-b74b-ff2e299bf859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:29.133215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e2bb7-9df4-4e62-b74b-ff2e299bf859 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:29.133499 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e2bb7-9df4-4e62-b74b-ff2e299bf859 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:29.133779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e2bb7-9df4-4e62-b74b-ff2e299bf859 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:29.134554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e2bb7-9df4-4e62-b74b-ff2e299bf859 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:29.135996 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e2bb7-9df4-4e62-b74b-ff2e299bf859 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:29.205177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.sql.core [None req-e21e2bb7-9df4-4e62-b74b-ff2e299bf859 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 10:03:29.205177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 10:03:29.205177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [parameters: {'user_id': 'e85bdc8854814ea5acc20dd3c4331ed2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 10:03:29.205177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80503) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 15 10:03:29.213410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-e21e2bb7-9df4-4e62-b74b-ff2e299bf859 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 10:03:29.213410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:29.213410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 10:03:29.213410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 10:03:29.213410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 10:03:29.213410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 10:03:29.213410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 10:03:29.213410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 10:03:29.213410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 10:03:29.213410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application conn.query(q) Jun 15 10:03:29.213410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 10:03:29.213410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 10:03:29.213410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 10:03:29.213410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application result.read() Jun 15 10:03:29.213410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 10:03:29.215582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application next(self.gen) Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application yield resource Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application next(self.gen) Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self.session.rollback() Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self.force_reraise() Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise self.value Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application session.commit() Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 15 10:03:29.216836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._prepare_impl() Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self.session.flush() Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._flush(objects) Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application compat.raise_( Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application flush_context.execute() Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rec.execute(self) Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 15 10:03:29.217891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 15 10:03:29.218778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application result = self._query(query) Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application conn.query(q) Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application result.read() Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 15 10:03:29.219610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 15 10:03:29.221432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 15 10:03:29.221432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 15 10:03:29.221432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 15 10:03:29.221432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 15 10:03:29.221432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 15 10:03:29.221432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 15 10:03:29.221432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e85bdc8854814ea5acc20dd3c4331ed2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 15 10:03:29.221432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 15 10:03:29.221432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:29.221432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:29.221432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:29.221432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:29.221432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:29.221432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:29.221432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 15 10:03:29.223285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 15 10:03:29.228431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 15 10:03:29.228431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 15 10:03:29.228431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 15 10:03:29.228431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 15 10:03:29.228431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 15 10:03:29.228431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:29.228431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 501/977] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Jun 15 10:03:28 2022] POST /identity/v3/users => generated 160 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:29.229377 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e965c-958b-4772-999a-4baec898e7b8 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:29.341782 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970498a7-3b41-47b1-bba5-6916f32cfa2d None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:29.342077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970498a7-3b41-47b1-bba5-6916f32cfa2d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:29.342367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970498a7-3b41-47b1-bba5-6916f32cfa2d None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:29.346859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 502/978] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:03:29.349843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e965c-958b-4772-999a-4baec898e7b8 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:29.352003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def408c3-90b0-4603-89aa-57b84ccfb28a None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:29.352299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def408c3-90b0-4603-89aa-57b84ccfb28a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:29.352581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def408c3-90b0-4603-89aa-57b84ccfb28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:29.370675 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-def408c3-90b0-4603-89aa-57b84ccfb28a None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:29.372615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-def408c3-90b0-4603-89aa-57b84ccfb28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:29.403713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1ae60-19ac-4fb0-bcda-f561459ed7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:29.404938 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1ae60-19ac-4fb0-bcda-f561459ed7cd None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:29.405379 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1ae60-19ac-4fb0-bcda-f561459ed7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:29.405793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1ae60-19ac-4fb0-bcda-f561459ed7cd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:29.415875 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1ae60-19ac-4fb0-bcda-f561459ed7cd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:29.418758 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1ae60-19ac-4fb0-bcda-f561459ed7cd None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-12c1ae60-19ac-4fb0-bcda-f561459ed7cd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:03:29.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:29.436025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:29.437456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:29.437456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:29.437456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:03:29.437456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:03:29.437456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:03:29.437456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:03:29.437456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:03:29.437456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:03:29.437456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 15 10:03:29.437456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:29.437456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 477/979] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:03:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:29.445959 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1ae60-19ac-4fb0-bcda-f561459ed7cd None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:29.453808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 503/980] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:29.474454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:29.476789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def408c3-90b0-4603-89aa-57b84ccfb28a None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:29.568277 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:29.573560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1ae60-19ac-4fb0-bcda-f561459ed7cd None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:29.607765 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def408c3-90b0-4603-89aa-57b84ccfb28a None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:29.621816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f6934-4f48-4c3e-85d5-e805d7438d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:29.623013 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f6934-4f48-4c3e-85d5-e805d7438d70 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:29.623502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f6934-4f48-4c3e-85d5-e805d7438d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:29.624383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f6934-4f48-4c3e-85d5-e805d7438d70 None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:29.625712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f6934-4f48-4c3e-85d5-e805d7438d70 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:29.626404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f6934-4f48-4c3e-85d5-e805d7438d70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:29.628423 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f6934-4f48-4c3e-85d5-e805d7438d70 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:29.651517 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ce684-285b-4ced-88df-36e7c4ec8488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:29.653172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ce684-285b-4ced-88df-36e7c4ec8488 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:29.653668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ce684-285b-4ced-88df-36e7c4ec8488 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:29.654168 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ce684-285b-4ced-88df-36e7c4ec8488 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:29.655127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ce684-285b-4ced-88df-36e7c4ec8488 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:29.656879 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ce684-285b-4ced-88df-36e7c4ec8488 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:29.676897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 478/981] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jun 15 10:03:29 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:29.723408 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 504/982] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jun 15 10:03:29 2022] POST /identity/v3/services => generated 242 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:03:30.393060 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a5b11-f20e-4fc7-89f1-5ce2ae803329 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:30.393399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a5b11-f20e-4fc7-89f1-5ce2ae803329 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:30.393638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a5b11-f20e-4fc7-89f1-5ce2ae803329 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:30.395191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 479/983] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:30.400526 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874de09-1e18-47ef-ac99-d1e33e273d1f None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:30.401016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874de09-1e18-47ef-ac99-d1e33e273d1f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:30.401209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874de09-1e18-47ef-ac99-d1e33e273d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:30.419142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-6874de09-1e18-47ef-ac99-d1e33e273d1f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:30.420352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6874de09-1e18-47ef-ac99-d1e33e273d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:30.510489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 505/984] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:30.535243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a5b11-f20e-4fc7-89f1-5ce2ae803329 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:30.632344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:30.654763 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a5b11-f20e-4fc7-89f1-5ce2ae803329 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:30.690397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4aef3-1188-460c-a9fb-024da7559120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:30.691199 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4aef3-1188-460c-a9fb-024da7559120 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:30.691479 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4aef3-1188-460c-a9fb-024da7559120 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:30.691751 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4aef3-1188-460c-a9fb-024da7559120 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:30.692402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4aef3-1188-460c-a9fb-024da7559120 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:30.693796 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4aef3-1188-460c-a9fb-024da7559120 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-23d4aef3-1188-460c-a9fb-024da7559120 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:30.707061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:30.708057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:30.708878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:30.708878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:30.708878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:30.708878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:30.708878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:30.708878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:30.708878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:30.708878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:30.708878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:30.708878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:30.708878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:30.708878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:30.708878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:03:30.708878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:30.708878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 480/985] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jun 15 10:03:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:30.716522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874de09-1e18-47ef-ac99-d1e33e273d1f None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:30.776443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:30.819230 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874de09-1e18-47ef-ac99-d1e33e273d1f None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:30.856185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa44e6d-f72a-432e-8456-f4a5f81277f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:30.857298 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa44e6d-f72a-432e-8456-f4a5f81277f0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:30.857583 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa44e6d-f72a-432e-8456-f4a5f81277f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:30.857860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa44e6d-f72a-432e-8456-f4a5f81277f0 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:30.858602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa44e6d-f72a-432e-8456-f4a5f81277f0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:30.858909 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa44e6d-f72a-432e-8456-f4a5f81277f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:30.860340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa44e6d-f72a-432e-8456-f4a5f81277f0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:30.870978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 506/986] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jun 15 10:03:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:30.872887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4aef3-1188-460c-a9fb-024da7559120 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:30.979561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4aef3-1188-460c-a9fb-024da7559120 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:31.018793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84ce2a-6df2-48ca-a201-4ede309db79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:31.019459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84ce2a-6df2-48ca-a201-4ede309db79d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:31.019740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84ce2a-6df2-48ca-a201-4ede309db79d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:31.020011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84ce2a-6df2-48ca-a201-4ede309db79d None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:31.020718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84ce2a-6df2-48ca-a201-4ede309db79d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:31.021044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84ce2a-6df2-48ca-a201-4ede309db79d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:31.022406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84ce2a-6df2-48ca-a201-4ede309db79d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:31.032604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 481/987] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jun 15 10:03:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:31.034226 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa44e6d-f72a-432e-8456-f4a5f81277f0 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:31.036244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb6500-c813-45e7-8f90-55cdc0d64bb3 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:31.036535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb6500-c813-45e7-8f90-55cdc0d64bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:31.036813 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb6500-c813-45e7-8f90-55cdc0d64bb3 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:31.038411 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 482/988] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:03:31.043747 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348326a-2a51-40ed-9204-c336f92bdedb None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:31.044041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348326a-2a51-40ed-9204-c336f92bdedb None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:31.044324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348326a-2a51-40ed-9204-c336f92bdedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:31.062313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-d348326a-2a51-40ed-9204-c336f92bdedb None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:31.063630 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d348326a-2a51-40ed-9204-c336f92bdedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:31.065892 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:31.136136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 483/989] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:31 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:31.143300 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa44e6d-f72a-432e-8456-f4a5f81277f0 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:31.159546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348326a-2a51-40ed-9204-c336f92bdedb None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:31.168657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:31.180524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4eb456-7bd4-4c08-b689-b68c0a7a98db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:31.181219 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4eb456-7bd4-4c08-b689-b68c0a7a98db None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:31.181499 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4eb456-7bd4-4c08-b689-b68c0a7a98db None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:31.181772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4eb456-7bd4-4c08-b689-b68c0a7a98db None admin] PATH_INFO: `/v3/endpoints` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:31.182566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4eb456-7bd4-4c08-b689-b68c0a7a98db None admin] RBAC: Adding query filter params (interface=admin, service_id=b7e0b41d1cf94a8a8d31f791d2a3507a, region_id=RegionOne) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:31.186142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4eb456-7bd4-4c08-b689-b68c0a7a98db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:31.186142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4eb456-7bd4-4c08-b689-b68c0a7a98db None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:31.190565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 507/990] 10.222.0.60 () {56 vars in 1249 bytes} [Wed Jun 15 10:03:31 2022] GET /identity/v3/endpoints?service_id=b7e0b41d1cf94a8a8d31f791d2a3507a&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:31.196008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4eb456-7bd4-4c08-b689-b68c0a7a98db None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:31.268549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348326a-2a51-40ed-9204-c336f92bdedb None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:31.309271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ff131-7f43-48e7-ab18-248bd7ff5b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:31.310229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ff131-7f43-48e7-ab18-248bd7ff5b2e None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:31.310544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ff131-7f43-48e7-ab18-248bd7ff5b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:31.310716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4eb456-7bd4-4c08-b689-b68c0a7a98db None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:31.310866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ff131-7f43-48e7-ab18-248bd7ff5b2e None admin] PATH_INFO: `/v3/services/image` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:31.311417 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ff131-7f43-48e7-ab18-248bd7ff5b2e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:31.314298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ff131-7f43-48e7-ab18-248bd7ff5b2e None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-4c8ff131-7f43-48e7-ab18-248bd7ff5b2e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:31.325580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:31.327532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:31.328365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:31.328365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:31.328365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:31.328365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:31.328365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:31.328365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:31.328365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:31.328365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:31.328365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:31.328365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:31.328365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:31.328365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:31.328365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:03:31.328365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:31.328365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 484/991] 10.222.0.60 () {56 vars in 1105 bytes} [Wed Jun 15 10:03:31 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:31.332468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ff131-7f43-48e7-ab18-248bd7ff5b2e None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:31.335481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:31.362045 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71136bef-ff92-4866-b120-5110149d2ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:31.362822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71136bef-ff92-4866-b120-5110149d2ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:31.363177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71136bef-ff92-4866-b120-5110149d2ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:31.363434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71136bef-ff92-4866-b120-5110149d2ae8 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:31.364064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71136bef-ff92-4866-b120-5110149d2ae8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:31.365565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71136bef-ff92-4866-b120-5110149d2ae8 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:31.373167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 508/992] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jun 15 10:03:31 2022] GET /identity/v3/services => generated 1727 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:03:31.454252 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ff131-7f43-48e7-ab18-248bd7ff5b2e None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:31.491710 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22335c4f-1e02-4260-97da-a7769a686e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:31.492400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22335c4f-1e02-4260-97da-a7769a686e5b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:31.492688 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22335c4f-1e02-4260-97da-a7769a686e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:31.492999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22335c4f-1e02-4260-97da-a7769a686e5b None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:31.493735 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22335c4f-1e02-4260-97da-a7769a686e5b None admin] RBAC: Adding query filter params (name=image) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:31.494049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22335c4f-1e02-4260-97da-a7769a686e5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:31.495469 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22335c4f-1e02-4260-97da-a7769a686e5b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:31.502037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 485/993] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:31.507782 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71136bef-ff92-4866-b120-5110149d2ae8 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:31.574423 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:31.627079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71136bef-ff92-4866-b120-5110149d2ae8 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:31.664773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854af6a6-4fc2-464d-b038-129106eabe18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:31.665483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854af6a6-4fc2-464d-b038-129106eabe18 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:31.665770 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854af6a6-4fc2-464d-b038-129106eabe18 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:31.666044 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854af6a6-4fc2-464d-b038-129106eabe18 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:31.666755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854af6a6-4fc2-464d-b038-129106eabe18 None admin] RBAC: Adding query filter params (type=image) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:31.667058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854af6a6-4fc2-464d-b038-129106eabe18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:31.669568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854af6a6-4fc2-464d-b038-129106eabe18 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:31.681014 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 509/994] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:31 2022] GET /identity/v3/services?type=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:31.944745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba912752-c3f3-4296-98e9-fec2cbd83ce0 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:31.945263 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba912752-c3f3-4296-98e9-fec2cbd83ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:31.945263 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba912752-c3f3-4296-98e9-fec2cbd83ce0 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:31.946795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 486/995] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:31.952781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88b6b7-57f8-4153-b76d-1a6d1a294e78 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:31.953121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88b6b7-57f8-4153-b76d-1a6d1a294e78 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:31.953402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88b6b7-57f8-4153-b76d-1a6d1a294e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:31.972796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-9b88b6b7-57f8-4153-b76d-1a6d1a294e78 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:31.974016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9b88b6b7-57f8-4153-b76d-1a6d1a294e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:32.045513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 510/996] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:31 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:32.069909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba912752-c3f3-4296-98e9-fec2cbd83ce0 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:32.140484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:32.179771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba912752-c3f3-4296-98e9-fec2cbd83ce0 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:32.216492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d527964-33f3-46d2-8df3-bef8ce6a8009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:32.217356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d527964-33f3-46d2-8df3-bef8ce6a8009 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:32.217639 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d527964-33f3-46d2-8df3-bef8ce6a8009 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:32.217914 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d527964-33f3-46d2-8df3-bef8ce6a8009 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:32.218685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d527964-33f3-46d2-8df3-bef8ce6a8009 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:32.220257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d527964-33f3-46d2-8df3-bef8ce6a8009 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-8d527964-33f3-46d2-8df3-bef8ce6a8009 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:32.234327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:32.235332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:32.236171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:32.236171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:32.236171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:32.236171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:32.236171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:32.236171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:32.236171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:32.236171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:32.236171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:32.236171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:32.236171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:32.236171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:32.236171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:03:32.236171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:32.243696 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 487/997] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:32.243837 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88b6b7-57f8-4153-b76d-1a6d1a294e78 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:32.289838 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:32.349060 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88b6b7-57f8-4153-b76d-1a6d1a294e78 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:32.384488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c3931-530f-4ed5-9753-f36433413651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:32.385189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c3931-530f-4ed5-9753-f36433413651 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:32.385474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c3931-530f-4ed5-9753-f36433413651 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:32.385748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c3931-530f-4ed5-9753-f36433413651 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:32.386460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c3931-530f-4ed5-9753-f36433413651 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:32.386803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c3931-530f-4ed5-9753-f36433413651 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:32.388191 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c3931-530f-4ed5-9753-f36433413651 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:32.393882 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 511/998] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jun 15 10:03:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:32.399344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d527964-33f3-46d2-8df3-bef8ce6a8009 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:32.507781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d527964-33f3-46d2-8df3-bef8ce6a8009 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:32.547085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:32.553517 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da812d6-2533-442c-b33d-17b779b31667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:32.554181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da812d6-2533-442c-b33d-17b779b31667 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:32.555887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da812d6-2533-442c-b33d-17b779b31667 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:32.555887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da812d6-2533-442c-b33d-17b779b31667 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:32.555887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da812d6-2533-442c-b33d-17b779b31667 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:32.555887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da812d6-2533-442c-b33d-17b779b31667 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:32.557091 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da812d6-2533-442c-b33d-17b779b31667 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:32.563086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 488/999] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jun 15 10:03:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:32.569424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c3931-530f-4ed5-9753-f36433413651 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:32.678725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c3931-530f-4ed5-9753-f36433413651 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:32.689403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:32.716305 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45498986-2235-4559-b256-08d4de9c8b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:32.716989 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45498986-2235-4559-b256-08d4de9c8b1e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:32.717277 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45498986-2235-4559-b256-08d4de9c8b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:32.717554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45498986-2235-4559-b256-08d4de9c8b1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:32.718349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45498986-2235-4559-b256-08d4de9c8b1e None admin] RBAC: Adding query filter params (interface=public, service_id=0bb8a88691ba4fa5ad3d3c17226f1a10, region_id=RegionOne) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:32.718659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45498986-2235-4559-b256-08d4de9c8b1e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:32.720128 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45498986-2235-4559-b256-08d4de9c8b1e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:32.731473 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 512/1000] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jun 15 10:03:32 2022] GET /identity/v3/endpoints?service_id=0bb8a88691ba4fa5ad3d3c17226f1a10&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:32.732469 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da812d6-2533-442c-b33d-17b779b31667 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:32.845429 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da812d6-2533-442c-b33d-17b779b31667 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:32.880987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7b7d7-e48e-476e-a894-dd479c5f71c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:32.881656 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7b7d7-e48e-476e-a894-dd479c5f71c8 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:32.881941 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7b7d7-e48e-476e-a894-dd479c5f71c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:32.882218 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7b7d7-e48e-476e-a894-dd479c5f71c8 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:32.882971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f7b7d7-e48e-476e-a894-dd479c5f71c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:32.884356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f7b7d7-e48e-476e-a894-dd479c5f71c8 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:32.891630 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 489/1001] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jun 15 10:03:32 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:03:33.642199 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9415f3-688d-4634-affb-790a84417cf3 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:33.642772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9415f3-688d-4634-affb-790a84417cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:33.642772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9415f3-688d-4634-affb-790a84417cf3 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:33.644118 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 513/1002] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:33.649857 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb79d7-57eb-4f5c-8912-9dcdc309a460 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:33.650318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb79d7-57eb-4f5c-8912-9dcdc309a460 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:33.650497 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb79d7-57eb-4f5c-8912-9dcdc309a460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:33.677850 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-24eb79d7-57eb-4f5c-8912-9dcdc309a460 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:33.679827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-24eb79d7-57eb-4f5c-8912-9dcdc309a460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:33.770103 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:33.786024 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 490/1003] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:33.810414 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9415f3-688d-4634-affb-790a84417cf3 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:33.921560 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9415f3-688d-4634-affb-790a84417cf3 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:33.956758 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b76102-2e10-43f7-93ee-04437301b64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:33.957700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b76102-2e10-43f7-93ee-04437301b64d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:33.957985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b76102-2e10-43f7-93ee-04437301b64d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:33.958259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b76102-2e10-43f7-93ee-04437301b64d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:33.958910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b76102-2e10-43f7-93ee-04437301b64d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:33.960288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b76102-2e10-43f7-93ee-04437301b64d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:33.963411 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c542c-e416-4953-8dd8-01d4e868093c None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:33.963702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c542c-e416-4953-8dd8-01d4e868093c None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:33.963983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c542c-e416-4953-8dd8-01d4e868093c None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:33.966278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 491/1004] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:33.971008 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018f3f6-51aa-4851-8c7f-72b11558eeb8 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:33.971297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018f3f6-51aa-4851-8c7f-72b11558eeb8 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:33.971576 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018f3f6-51aa-4851-8c7f-72b11558eeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-f6b76102-2e10-43f7-93ee-04437301b64d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:33.973227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:33.974364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:33.975525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:33.975525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:33.975525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:33.975525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:33.975525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:33.975525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:33.975525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:33.975525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:33.975525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:33.975525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:33.975525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:33.975525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:33.975525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 15 10:03:33.975525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:33.979305 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 514/1005] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jun 15 10:03:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:33.980164 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b76102-2e10-43f7-93ee-04437301b64d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:33.995309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-e018f3f6-51aa-4851-8c7f-72b11558eeb8 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:33.996739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e018f3f6-51aa-4851-8c7f-72b11558eeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:34.017046 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:34.074043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 492/1006] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:34.092692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b76102-2e10-43f7-93ee-04437301b64d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:34.099546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018f3f6-51aa-4851-8c7f-72b11558eeb8 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:34.135673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adafa43-f39b-44ed-ab55-16d4dc99a18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:34.136358 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adafa43-f39b-44ed-ab55-16d4dc99a18d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:34.137314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adafa43-f39b-44ed-ab55-16d4dc99a18d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:34.137647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adafa43-f39b-44ed-ab55-16d4dc99a18d None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:34.138377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adafa43-f39b-44ed-ab55-16d4dc99a18d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:34.138684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adafa43-f39b-44ed-ab55-16d4dc99a18d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:34.140694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adafa43-f39b-44ed-ab55-16d4dc99a18d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:34.148634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 515/1007] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jun 15 10:03:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:34.154553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adafa43-f39b-44ed-ab55-16d4dc99a18d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:34.210270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018f3f6-51aa-4851-8c7f-72b11558eeb8 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:34.229466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:34.247731 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350c35e-21cd-44bf-9b6f-6077c6ac7389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:34.248403 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350c35e-21cd-44bf-9b6f-6077c6ac7389 None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:34.248692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350c35e-21cd-44bf-9b6f-6077c6ac7389 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:34.248986 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350c35e-21cd-44bf-9b6f-6077c6ac7389 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:34.249740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350c35e-21cd-44bf-9b6f-6077c6ac7389 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:34.251183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350c35e-21cd-44bf-9b6f-6077c6ac7389 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:34.259224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:34.263233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adafa43-f39b-44ed-ab55-16d4dc99a18d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:34.299398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a502e-ac3f-4be1-99e9-6ce268ef17f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:34.300378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a502e-ac3f-4be1-99e9-6ce268ef17f3 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:34.300665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a502e-ac3f-4be1-99e9-6ce268ef17f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:34.300945 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a502e-ac3f-4be1-99e9-6ce268ef17f3 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:34.301719 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a502e-ac3f-4be1-99e9-6ce268ef17f3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:34.302077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a502e-ac3f-4be1-99e9-6ce268ef17f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:34.303539 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a502e-ac3f-4be1-99e9-6ce268ef17f3 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:34.313212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 516/1008] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jun 15 10:03:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:34.314214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a502e-ac3f-4be1-99e9-6ce268ef17f3 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:34.328034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 493/1009] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jun 15 10:03:34 2022] POST /identity/v3/services => generated 242 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:03:34.426928 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a502e-ac3f-4be1-99e9-6ce268ef17f3 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:34.472813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8243d75-fa2a-4880-9b23-475c5da50eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:34.473579 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8243d75-fa2a-4880-9b23-475c5da50eac None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:34.473867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8243d75-fa2a-4880-9b23-475c5da50eac None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:34.474167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8243d75-fa2a-4880-9b23-475c5da50eac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:34.474930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8243d75-fa2a-4880-9b23-475c5da50eac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:34.476334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8243d75-fa2a-4880-9b23-475c5da50eac None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:34.551372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 517/1010] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jun 15 10:03:34 2022] POST /identity/v3/endpoints => generated 325 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:03:35.143101 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba99f3d-9b97-498e-a700-dec8998e134e None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:35.143621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba99f3d-9b97-498e-a700-dec8998e134e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:35.143723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba99f3d-9b97-498e-a700-dec8998e134e None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:35.145032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 494/1011] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:35.150367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3c6be-9c79-4043-ae43-d1d5dce7a1b0 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:35.150793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3c6be-9c79-4043-ae43-d1d5dce7a1b0 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:35.150963 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3c6be-9c79-4043-ae43-d1d5dce7a1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:35.168862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-90c3c6be-9c79-4043-ae43-d1d5dce7a1b0 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:35.170209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-90c3c6be-9c79-4043-ae43-d1d5dce7a1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:35.185870 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:35.246683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 518/1012] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:35 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:35.269292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba99f3d-9b97-498e-a700-dec8998e134e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:35.376358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba99f3d-9b97-498e-a700-dec8998e134e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:35.412073 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bc1ea-f9c7-46b8-b143-f0384255fe0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:35.413186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bc1ea-f9c7-46b8-b143-f0384255fe0d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:35.413672 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bc1ea-f9c7-46b8-b143-f0384255fe0d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:35.414136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bc1ea-f9c7-46b8-b143-f0384255fe0d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:35.415050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bc1ea-f9c7-46b8-b143-f0384255fe0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:35.416577 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bc1ea-f9c7-46b8-b143-f0384255fe0d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-276bc1ea-f9c7-46b8-b143-f0384255fe0d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:35.429856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:35.430852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:35.431684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:35.431684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:35.431684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:35.431684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:35.431684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:35.431684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:35.431684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:35.431684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:35.431684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:35.431684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:35.431684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:35.431684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:35.431684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 15 10:03:35.431684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:35.438662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 495/1013] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:35.439950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3c6be-9c79-4043-ae43-d1d5dce7a1b0 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:35.552434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3c6be-9c79-4043-ae43-d1d5dce7a1b0 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:35.587348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dbab7-b0b4-4f8a-9f7d-63d1268d6ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:35.588017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dbab7-b0b4-4f8a-9f7d-63d1268d6ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:35.588301 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dbab7-b0b4-4f8a-9f7d-63d1268d6ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:35.588578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093dbab7-b0b4-4f8a-9f7d-63d1268d6ae0 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:35.589321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093dbab7-b0b4-4f8a-9f7d-63d1268d6ae0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:35.590204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093dbab7-b0b4-4f8a-9f7d-63d1268d6ae0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:35.591687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093dbab7-b0b4-4f8a-9f7d-63d1268d6ae0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:35.601526 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 519/1014] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jun 15 10:03:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:35.602478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bc1ea-f9c7-46b8-b143-f0384255fe0d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:35.647337 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:35.709537 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bc1ea-f9c7-46b8-b143-f0384255fe0d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:35.744361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae9f121-9886-4af2-91e3-69f72dc78775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:35.745060 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9f121-9886-4af2-91e3-69f72dc78775 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:35.745345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9f121-9886-4af2-91e3-69f72dc78775 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:35.745618 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae9f121-9886-4af2-91e3-69f72dc78775 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:35.746568 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9f121-9886-4af2-91e3-69f72dc78775 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:35.746877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9f121-9886-4af2-91e3-69f72dc78775 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:35.748243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae9f121-9886-4af2-91e3-69f72dc78775 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:35.758367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 496/1015] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jun 15 10:03:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:35.759327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dbab7-b0b4-4f8a-9f7d-63d1268d6ae0 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:35.772449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:35.862961 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093dbab7-b0b4-4f8a-9f7d-63d1268d6ae0 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:35.897386 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee672e6-45ed-41a1-b3cd-43307f7865b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:35.898050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee672e6-45ed-41a1-b3cd-43307f7865b7 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:35.898338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee672e6-45ed-41a1-b3cd-43307f7865b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:35.898615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee672e6-45ed-41a1-b3cd-43307f7865b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:35.899372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee672e6-45ed-41a1-b3cd-43307f7865b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:35.900747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee672e6-45ed-41a1-b3cd-43307f7865b7 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:35.976923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 520/1016] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jun 15 10:03:35 2022] POST /identity/v3/endpoints => generated 334 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:03:36.527205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15717391-c0a0-4b1a-b326-d339c9e2be12 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:36.527746 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15717391-c0a0-4b1a-b326-d339c9e2be12 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:36.527746 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15717391-c0a0-4b1a-b326-d339c9e2be12 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:36.529313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 497/1017] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:36.534493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bffde-09f0-426a-b728-08393199d743 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:36.534915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bffde-09f0-426a-b728-08393199d743 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:36.535123 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bffde-09f0-426a-b728-08393199d743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:36.552792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-f53bffde-09f0-426a-b728-08393199d743 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:36.553994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f53bffde-09f0-426a-b728-08393199d743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:36.617858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:36.634187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 521/1018] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:36.655467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15717391-c0a0-4b1a-b326-d339c9e2be12 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:36.766921 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15717391-c0a0-4b1a-b326-d339c9e2be12 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:36.774203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:36.775233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f4c72-26b5-428b-bd54-02f38e0131a8 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:36.775524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f4c72-26b5-428b-bd54-02f38e0131a8 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:36.775809 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f4c72-26b5-428b-bd54-02f38e0131a8 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:36.777933 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 522/1019] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:03:36.783540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1220c-1a62-4ac4-a6be-286cc83afd09 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:36.783828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1220c-1a62-4ac4-a6be-286cc83afd09 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:36.784108 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1220c-1a62-4ac4-a6be-286cc83afd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:36.801308 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-f3d1220c-1a62-4ac4-a6be-286cc83afd09 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:36.802481 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f3d1220c-1a62-4ac4-a6be-286cc83afd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:36.804217 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06e85f-1b6f-42d5-ae85-d21e142500b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:36.806201 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06e85f-1b6f-42d5-ae85-d21e142500b3 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:36.806681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06e85f-1b6f-42d5-ae85-d21e142500b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:36.806978 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06e85f-1b6f-42d5-ae85-d21e142500b3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:36.808052 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06e85f-1b6f-42d5-ae85-d21e142500b3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:36.810419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06e85f-1b6f-42d5-ae85-d21e142500b3 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-6f06e85f-1b6f-42d5-ae85-d21e142500b3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:36.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:36.828378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:36.839066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:36.839066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:36.839066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:36.839066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:36.839066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:36.839066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:36.839066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:36.839066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:36.839066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:36.839066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:36.839066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:36.839066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:36.839066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:03:36.839066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:36.839066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 498/1020] 10.222.0.60 () {56 vars in 1105 bytes} [Wed Jun 15 10:03:36 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:36.841181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06e85f-1b6f-42d5-ae85-d21e142500b3 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:36.882297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 523/1021] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:36.906864 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1220c-1a62-4ac4-a6be-286cc83afd09 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:36.950981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06e85f-1b6f-42d5-ae85-d21e142500b3 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:36.987084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f93f6-39f0-4134-ad8b-7c5ee3a7e8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:36.987755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f93f6-39f0-4134-ad8b-7c5ee3a7e8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:36.988041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f93f6-39f0-4134-ad8b-7c5ee3a7e8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:36.988324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f93f6-39f0-4134-ad8b-7c5ee3a7e8c5 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:36.989072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f93f6-39f0-4134-ad8b-7c5ee3a7e8c5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:36.989386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f93f6-39f0-4134-ad8b-7c5ee3a7e8c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:36.990755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f93f6-39f0-4134-ad8b-7c5ee3a7e8c5 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:37.000753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 499/1022] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:37.001678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f93f6-39f0-4134-ad8b-7c5ee3a7e8c5 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:37.015385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1220c-1a62-4ac4-a6be-286cc83afd09 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:37.038196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:37.041469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:37.054747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661d3a6-a584-4d9f-b607-c706f32ffb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:37.055551 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661d3a6-a584-4d9f-b607-c706f32ffb7c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:37.055837 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661d3a6-a584-4d9f-b607-c706f32ffb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:37.056198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661d3a6-a584-4d9f-b607-c706f32ffb7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:37.057431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661d3a6-a584-4d9f-b607-c706f32ffb7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:37.059417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661d3a6-a584-4d9f-b607-c706f32ffb7c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:37.061391 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 524/1023] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:37.067093 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661d3a6-a584-4d9f-b607-c706f32ffb7c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:37.117204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f93f6-39f0-4134-ad8b-7c5ee3a7e8c5 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:37.153455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c69ca6-2924-4b4d-9e43-9d5ee92dac10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:37.154301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c69ca6-2924-4b4d-9e43-9d5ee92dac10 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:37.154584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c69ca6-2924-4b4d-9e43-9d5ee92dac10 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:37.154857 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c69ca6-2924-4b4d-9e43-9d5ee92dac10 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:37.155570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c69ca6-2924-4b4d-9e43-9d5ee92dac10 None admin] RBAC: Adding query filter params (type=image) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:37.155874 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c69ca6-2924-4b4d-9e43-9d5ee92dac10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:37.157255 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c69ca6-2924-4b4d-9e43-9d5ee92dac10 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:37.162815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 500/1024] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:37.169464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c69ca6-2924-4b4d-9e43-9d5ee92dac10 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:37.179834 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661d3a6-a584-4d9f-b607-c706f32ffb7c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:37.222134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf79fd-88d4-4072-b745-48770f7d3bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:37.222810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf79fd-88d4-4072-b745-48770f7d3bcb None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:37.223095 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf79fd-88d4-4072-b745-48770f7d3bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:37.223373 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf79fd-88d4-4072-b745-48770f7d3bcb None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:37.224125 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf79fd-88d4-4072-b745-48770f7d3bcb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:37.225618 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf79fd-88d4-4072-b745-48770f7d3bcb None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:37.279893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:37.283927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c69ca6-2924-4b4d-9e43-9d5ee92dac10 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:37.318925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c6c0c-f8c2-4c6b-b387-7848c17403b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:37.319648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c6c0c-f8c2-4c6b-b387-7848c17403b7 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:37.319937 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c6c0c-f8c2-4c6b-b387-7848c17403b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:37.320217 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c6c0c-f8c2-4c6b-b387-7848c17403b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:37.321078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c6c0c-f8c2-4c6b-b387-7848c17403b7 None admin] RBAC: Adding query filter params (interface=public, service_id=599140eb7d5044cf96dd361f255dec29, region_id=RegionOne) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:37.321399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c6c0c-f8c2-4c6b-b387-7848c17403b7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:37.322795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c6c0c-f8c2-4c6b-b387-7848c17403b7 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:37.328507 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 501/1025] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jun 15 10:03:37 2022] GET /identity/v3/endpoints?service_id=599140eb7d5044cf96dd361f255dec29&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:37.330779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 525/1026] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jun 15 10:03:37 2022] POST /identity/v3/projects => generated 311 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:03:37.333770 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c6c0c-f8c2-4c6b-b387-7848c17403b7 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:37.452590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c6c0c-f8c2-4c6b-b387-7848c17403b7 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:37.490454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243be0f-5f0f-4f02-9a12-a88e4c058c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:37.491127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243be0f-5f0f-4f02-9a12-a88e4c058c18 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:37.491407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243be0f-5f0f-4f02-9a12-a88e4c058c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:37.491683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243be0f-5f0f-4f02-9a12-a88e4c058c18 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:37.492385 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243be0f-5f0f-4f02-9a12-a88e4c058c18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:37.493779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2243be0f-5f0f-4f02-9a12-a88e4c058c18 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:37.501588 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 502/1027] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jun 15 10:03:37 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:03:38.260860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698eaad0-cc86-47cc-bc66-102163086596 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:38.261414 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698eaad0-cc86-47cc-bc66-102163086596 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:38.261693 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698eaad0-cc86-47cc-bc66-102163086596 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:38.263663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 526/1028] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:38.268892 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512c96b-636f-4a40-92df-9360be413f37 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:38.269347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512c96b-636f-4a40-92df-9360be413f37 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:38.269543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512c96b-636f-4a40-92df-9360be413f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:38.286291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-e512c96b-636f-4a40-92df-9360be413f37 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:38.287481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e512c96b-636f-4a40-92df-9360be413f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:38.310526 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:38.359265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 503/1029] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:38.382616 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698eaad0-cc86-47cc-bc66-102163086596 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:38.431316 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:38.494125 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698eaad0-cc86-47cc-bc66-102163086596 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:38.533092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089f479-d6b6-4db6-b195-f445d4609a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:38.533846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089f479-d6b6-4db6-b195-f445d4609a2d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:38.534134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089f479-d6b6-4db6-b195-f445d4609a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:38.534415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089f479-d6b6-4db6-b195-f445d4609a2d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:38.549101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089f479-d6b6-4db6-b195-f445d4609a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-2089f479-d6b6-4db6-b195-f445d4609a2d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:03:38.557651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:38.559184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:38.560609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:38.560609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:38.560609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:38.560609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:38.560609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:03:38.560609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:03:38.560609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:03:38.560609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:03:38.560609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:03:38.560609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:03:38.560609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:03:38.560609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:38.560609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 527/1030] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Jun 15 10:03:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:38.565081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512c96b-636f-4a40-92df-9360be413f37 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:38.673863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512c96b-636f-4a40-92df-9360be413f37 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:38.714137 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624ce66-c6f1-4785-ad11-23f623d40409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:38.715442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624ce66-c6f1-4785-ad11-23f623d40409 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:38.715726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624ce66-c6f1-4785-ad11-23f623d40409 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:38.716104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624ce66-c6f1-4785-ad11-23f623d40409 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:38.716876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624ce66-c6f1-4785-ad11-23f623d40409 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:38.717219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624ce66-c6f1-4785-ad11-23f623d40409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:38.718599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624ce66-c6f1-4785-ad11-23f623d40409 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:38.732045 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 504/1031] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Jun 15 10:03:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:38.733654 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089f479-d6b6-4db6-b195-f445d4609a2d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:38.835177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089f479-d6b6-4db6-b195-f445d4609a2d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:38.836617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:38.868857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857fb907-c89e-4a74-aca0-099f36df80e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:38.869681 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fb907-c89e-4a74-aca0-099f36df80e3 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:38.869967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fb907-c89e-4a74-aca0-099f36df80e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:38.870244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fb907-c89e-4a74-aca0-099f36df80e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:38.882063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857fb907-c89e-4a74-aca0-099f36df80e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:38.883509 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857fb907-c89e-4a74-aca0-099f36df80e3 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:38.900978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-857fb907-c89e-4a74-aca0-099f36df80e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:38.900978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:38.900978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:38.900978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:38.900978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:38.900978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:38.900978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:38.900978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:38.900978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:38.900978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:38.900978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:38.900978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:38.900978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:38.900978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:38.900978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:38.901930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:38.902810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:38.902810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:38.902810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:38.902810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:38.907267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 528/1032] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:38.908632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624ce66-c6f1-4785-ad11-23f623d40409 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:38.930330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:39.021702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624ce66-c6f1-4785-ad11-23f623d40409 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:39.059471 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb5484-c799-4d1e-a847-8fd5e4a772a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:39.060203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb5484-c799-4d1e-a847-8fd5e4a772a9 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:39.060481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb5484-c799-4d1e-a847-8fd5e4a772a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:39.060753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb5484-c799-4d1e-a847-8fd5e4a772a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:39.061606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb5484-c799-4d1e-a847-8fd5e4a772a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:39.061918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb5484-c799-4d1e-a847-8fd5e4a772a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:39.063437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb5484-c799-4d1e-a847-8fd5e4a772a9 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:39.081062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 505/1033] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:39.082659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857fb907-c89e-4a74-aca0-099f36df80e3 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:39.189621 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857fb907-c89e-4a74-aca0-099f36df80e3 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:39.224682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc225d1-d6fc-4d4c-9f73-fd17446115f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:39.225510 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc225d1-d6fc-4d4c-9f73-fd17446115f1 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:39.225798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc225d1-d6fc-4d4c-9f73-fd17446115f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:39.226073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc225d1-d6fc-4d4c-9f73-fd17446115f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:39.234008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc225d1-d6fc-4d4c-9f73-fd17446115f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:39.235459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc225d1-d6fc-4d4c-9f73-fd17446115f1 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-1cc225d1-d6fc-4d4c-9f73-fd17446115f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:03:39.250557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:39.251568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:39.252520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:39.252520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:39.252520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:03:39.252520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:03:39.252520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:03:39.252520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:03:39.252520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:03:39.252520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:03:39.252520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:03:39.252520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:39.252520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 529/1034] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jun 15 10:03:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:39.259209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb5484-c799-4d1e-a847-8fd5e4a772a9 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:39.341195 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:39.364121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb5484-c799-4d1e-a847-8fd5e4a772a9 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:39.399767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f5888-3016-4398-9903-ede3be1db17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:39.400683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f5888-3016-4398-9903-ede3be1db17c None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:39.401137 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f5888-3016-4398-9903-ede3be1db17c None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:39.401559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f5888-3016-4398-9903-ede3be1db17c None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:39.403139 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f5888-3016-4398-9903-ede3be1db17c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:39.403593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f5888-3016-4398-9903-ede3be1db17c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:39.405184 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f5888-3016-4398-9903-ede3be1db17c None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:39.423438 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 506/1035] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jun 15 10:03:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:39.429356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc225d1-d6fc-4d4c-9f73-fd17446115f1 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:39.446363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:39.548645 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbfc2fb-e27f-4883-9311-70b67b21c30f None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:39.548940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbfc2fb-e27f-4883-9311-70b67b21c30f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:39.549235 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc225d1-d6fc-4d4c-9f73-fd17446115f1 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:39.549339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbfc2fb-e27f-4883-9311-70b67b21c30f None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:39.551144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 507/1036] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:03:39.556049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bd7f5-ff15-4fe9-8896-6e4964d33c89 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:39.556342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bd7f5-ff15-4fe9-8896-6e4964d33c89 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:39.556626 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bd7f5-ff15-4fe9-8896-6e4964d33c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:39.572906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-c56bd7f5-ff15-4fe9-8896-6e4964d33c89 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:39.574714 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c56bd7f5-ff15-4fe9-8896-6e4964d33c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:39.585998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc40bd-6133-49b8-b2c2-a81968972006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:39.586668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc40bd-6133-49b8-b2c2-a81968972006 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:39.586959 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc40bd-6133-49b8-b2c2-a81968972006 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:39.587246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc40bd-6133-49b8-b2c2-a81968972006 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:39.653340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 508/1037] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:39.664634 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839dd40-c1c6-48c9-bf0c-8942b9367163 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:39.665155 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839dd40-c1c6-48c9-bf0c-8942b9367163 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:39.665443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839dd40-c1c6-48c9-bf0c-8942b9367163 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:39.666652 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc40bd-6133-49b8-b2c2-a81968972006 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:39.668990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 509/1038] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:03:39.669136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc40bd-6133-49b8-b2c2-a81968972006 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:39.673198 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b822d-14eb-4f46-86ea-fd8870a67262 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:39.673491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b822d-14eb-4f46-86ea-fd8870a67262 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:39.673834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b822d-14eb-4f46-86ea-fd8870a67262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:39.684370 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:39.693029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-0b1b822d-14eb-4f46-86ea-fd8870a67262 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:39.694214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-0b1b822d-14eb-4f46-86ea-fd8870a67262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:39.756249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:39.764114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 510/1039] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:39.765043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b822d-14eb-4f46-86ea-fd8870a67262 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:39.773015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 530/1040] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:03:39 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:03:39.778849 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc40bd-6133-49b8-b2c2-a81968972006 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:39.864895 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b822d-14eb-4f46-86ea-fd8870a67262 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:39.880576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc40bd-6133-49b8-b2c2-a81968972006 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:39.895775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:39.902355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6eb7b-43db-4e62-8d8e-d95fe4749283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:39.911765 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6eb7b-43db-4e62-8d8e-d95fe4749283 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:39.911765 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6eb7b-43db-4e62-8d8e-d95fe4749283 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:39.911765 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6eb7b-43db-4e62-8d8e-d95fe4749283 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:39.911765 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6eb7b-43db-4e62-8d8e-d95fe4749283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:39.911765 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6eb7b-43db-4e62-8d8e-d95fe4749283 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:39.911765 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 511/1041] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:39.912216 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6eb7b-43db-4e62-8d8e-d95fe4749283 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:39.919127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f50c8b-477d-4fe3-8f5e-f73baa23e244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:39.919943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f50c8b-477d-4fe3-8f5e-f73baa23e244 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:39.920225 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f50c8b-477d-4fe3-8f5e-f73baa23e244 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:39.920503 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f50c8b-477d-4fe3-8f5e-f73baa23e244 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:39.932089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f50c8b-477d-4fe3-8f5e-f73baa23e244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:39.933822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f50c8b-477d-4fe3-8f5e-f73baa23e244 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:39.951540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-15f50c8b-477d-4fe3-8f5e-f73baa23e244 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:39.951540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:39.951540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:39.951540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:39.951540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:39.951540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:39.951540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:39.951540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:39.951540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:39.951540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:39.951540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:39.951540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:39.951540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:39.951540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:39.951540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:39.952482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:39.953437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:39.953437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:39.953437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:39.953437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:39.953437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 531/1042] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:39.954217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f50c8b-477d-4fe3-8f5e-f73baa23e244 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:39.978100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:40.016076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6eb7b-43db-4e62-8d8e-d95fe4749283 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:40.051575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3dbd5-27cb-4f6a-89aa-0a359e7f0b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:40.052359 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3dbd5-27cb-4f6a-89aa-0a359e7f0b72 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:40.052640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3dbd5-27cb-4f6a-89aa-0a359e7f0b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:40.052920 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3dbd5-27cb-4f6a-89aa-0a359e7f0b72 None admin] PATH_INFO: `/v3/services/image` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:40.053592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3dbd5-27cb-4f6a-89aa-0a359e7f0b72 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:40.053705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f50c8b-477d-4fe3-8f5e-f73baa23e244 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:40.054953 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3dbd5-27cb-4f6a-89aa-0a359e7f0b72 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-30e3dbd5-27cb-4f6a-89aa-0a359e7f0b72 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:40.067080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:40.068317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:40.069441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:40.069441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:40.069441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:40.069441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:40.069441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:40.069441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:40.069441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:40.069441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:40.069441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:40.069441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:40.069441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:03:40.069441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:03:40.069441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 15 10:03:40.069441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:40.069441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 512/1043] 10.222.0.60 () {56 vars in 1105 bytes} [Wed Jun 15 10:03:39 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:40.070419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3dbd5-27cb-4f6a-89aa-0a359e7f0b72 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:40.091749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8888fbd-1adf-4ea2-95cd-91d7aabc9666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:40.092425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8888fbd-1adf-4ea2-95cd-91d7aabc9666 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:40.092709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8888fbd-1adf-4ea2-95cd-91d7aabc9666 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:40.093074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8888fbd-1adf-4ea2-95cd-91d7aabc9666 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:40.093790 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8888fbd-1adf-4ea2-95cd-91d7aabc9666 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:40.095225 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8888fbd-1adf-4ea2-95cd-91d7aabc9666 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:40.173209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3dbd5-27cb-4f6a-89aa-0a359e7f0b72 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:40.177807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 532/1044] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:03:39 2022] POST /identity/v3/users => generated 285 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:03:40.181887 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8888fbd-1adf-4ea2-95cd-91d7aabc9666 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:40.208606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:40.212344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:40.221942 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bc37b-b887-41d7-b96f-9da4b5ef1461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:40.222657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bc37b-b887-41d7-b96f-9da4b5ef1461 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:40.222952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bc37b-b887-41d7-b96f-9da4b5ef1461 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:40.223219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bc37b-b887-41d7-b96f-9da4b5ef1461 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:40.223981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bc37b-b887-41d7-b96f-9da4b5ef1461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:40.224280 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bc37b-b887-41d7-b96f-9da4b5ef1461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:40.225726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bc37b-b887-41d7-b96f-9da4b5ef1461 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:40.240563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 513/1045] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:40.247157 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bc37b-b887-41d7-b96f-9da4b5ef1461 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:40.307561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8888fbd-1adf-4ea2-95cd-91d7aabc9666 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:40.347709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91888f5c-b730-4ab5-b314-d97316011eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:40.348393 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91888f5c-b730-4ab5-b314-d97316011eae None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:40.349309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91888f5c-b730-4ab5-b314-d97316011eae None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:40.349309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91888f5c-b730-4ab5-b314-d97316011eae None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:40.349698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91888f5c-b730-4ab5-b314-d97316011eae None admin] RBAC: Adding query filter params (name=image) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:40.350627 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91888f5c-b730-4ab5-b314-d97316011eae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:40.351750 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91888f5c-b730-4ab5-b314-d97316011eae None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:40.358230 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 533/1046] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:40.363671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91888f5c-b730-4ab5-b314-d97316011eae None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:40.369518 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bc37b-b887-41d7-b96f-9da4b5ef1461 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:40.416173 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889e33e-e249-4669-9b82-8dc2e945e7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:40.417363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889e33e-e249-4669-9b82-8dc2e945e7ea None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:40.418545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889e33e-e249-4669-9b82-8dc2e945e7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:40.418985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889e33e-e249-4669-9b82-8dc2e945e7ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:40.426704 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889e33e-e249-4669-9b82-8dc2e945e7ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:40.428148 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889e33e-e249-4669-9b82-8dc2e945e7ea None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:40.443065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-e889e33e-e249-4669-9b82-8dc2e945e7ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:40.443065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:40.443065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:40.443065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:40.443065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:40.443065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:40.443065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:40.443065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:40.443065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:40.443065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:40.443065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:40.443065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:40.443065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:03:40.443065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:03:40.443065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:40.444028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:40.444919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:03:40.444919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:03:40.444919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:03:40.444919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:03:40.444919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:03:40.444919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:03:40.444919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:40.444919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:40.449139 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 514/1047] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:40.450460 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889e33e-e249-4669-9b82-8dc2e945e7ea None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:40.468515 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:40.484215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91888f5c-b730-4ab5-b314-d97316011eae None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:40.503182 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:40.520735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83f67b-fb79-4864-b205-1c6c0d36e2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:40.521444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f67b-fb79-4864-b205-1c6c0d36e2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:40.521727 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f67b-fb79-4864-b205-1c6c0d36e2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:40.522008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83f67b-fb79-4864-b205-1c6c0d36e2b6 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:40.522782 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83f67b-fb79-4864-b205-1c6c0d36e2b6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:40.523099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83f67b-fb79-4864-b205-1c6c0d36e2b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:40.524491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83f67b-fb79-4864-b205-1c6c0d36e2b6 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:40.532242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 534/1048] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:40.537894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83f67b-fb79-4864-b205-1c6c0d36e2b6 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:40.565245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889e33e-e249-4669-9b82-8dc2e945e7ea None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:40.601982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3d708-e134-471f-b965-1c215004bdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:40.602591 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3d708-e134-471f-b965-1c215004bdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:40.602877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3d708-e134-471f-b965-1c215004bdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:40.603151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3d708-e134-471f-b965-1c215004bdb1 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:40.603944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3d708-e134-471f-b965-1c215004bdb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:40.604463 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3d708-e134-471f-b965-1c215004bdb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:40.606262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3d708-e134-471f-b965-1c215004bdb1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:40.619493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 515/1049] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:03:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:40.626066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3d708-e134-471f-b965-1c215004bdb1 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:40.655402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83f67b-fb79-4864-b205-1c6c0d36e2b6 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:40.697668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4c37b-33d1-4490-a5f7-3d5243fb0f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:40.698251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4c37b-33d1-4490-a5f7-3d5243fb0f89 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:40.698540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4c37b-33d1-4490-a5f7-3d5243fb0f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:40.699640 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4c37b-33d1-4490-a5f7-3d5243fb0f89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:40.700391 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4c37b-33d1-4490-a5f7-3d5243fb0f89 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:40.703042 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4c37b-33d1-4490-a5f7-3d5243fb0f89 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:40.744973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3d708-e134-471f-b965-1c215004bdb1 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:40.754608 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:40.779294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8385f9-fbea-4496-b2fc-b21a54c4c533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:40.779954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8385f9-fbea-4496-b2fc-b21a54c4c533 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:40.780241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8385f9-fbea-4496-b2fc-b21a54c4c533 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:40.780521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8385f9-fbea-4496-b2fc-b21a54c4c533 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:40.781400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8385f9-fbea-4496-b2fc-b21a54c4c533 None admin] RBAC: Adding query filter params (role.id=b1c676dcdeaa4ac0a2a3a7ef96317fa0, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=26f23f8595fc43ad94aee7d0192164a1) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:40.781747 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8385f9-fbea-4496-b2fc-b21a54c4c533 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:40.783164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8385f9-fbea-4496-b2fc-b21a54c4c533 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:40.785533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 535/1050] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jun 15 10:03:40 2022] POST /identity/v3/endpoints => generated 327 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:03:40.789011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 516/1051] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:03:40 2022] GET /identity/v3/role_assignments?user.id=26f23f8595fc43ad94aee7d0192164a1&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=b1c676dcdeaa4ac0a2a3a7ef96317fa0 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:42.438533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d77b4c-3921-4b3c-88a3-9dd6f38e8ada None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:42.439060 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d77b4c-3921-4b3c-88a3-9dd6f38e8ada None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:42.439172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d77b4c-3921-4b3c-88a3-9dd6f38e8ada None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:42.440513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 536/1052] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:42.445714 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f7ba7-ef99-4d2a-8e8b-c02493678a99 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:42.446125 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f7ba7-ef99-4d2a-8e8b-c02493678a99 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:42.446307 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f7ba7-ef99-4d2a-8e8b-c02493678a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:42.464131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-d58f7ba7-ef99-4d2a-8e8b-c02493678a99 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:42.466247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d58f7ba7-ef99-4d2a-8e8b-c02493678a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:42.552462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 517/1053] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:42.577406 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d77b4c-3921-4b3c-88a3-9dd6f38e8ada None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:42.587625 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:42.685902 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d77b4c-3921-4b3c-88a3-9dd6f38e8ada None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:42.721837 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cce94-0837-4ee6-945d-c4158cd9bb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:42.722620 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cce94-0837-4ee6-945d-c4158cd9bb24 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:42.722906 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cce94-0837-4ee6-945d-c4158cd9bb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:42.723182 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cce94-0837-4ee6-945d-c4158cd9bb24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:42.734893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505cce94-0837-4ee6-945d-c4158cd9bb24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-505cce94-0837-4ee6-945d-c4158cd9bb24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:03:42.742659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:42.743572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:42.744411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:42.744411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:42.744411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:42.744411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:42.744411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:03:42.744411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:03:42.744411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:03:42.744411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:03:42.744411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:03:42.744411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:03:42.744411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:03:42.744411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:42.749299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 537/1054] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:03:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:42.750629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f7ba7-ef99-4d2a-8e8b-c02493678a99 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:42.849460 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:42.853825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f7ba7-ef99-4d2a-8e8b-c02493678a99 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:42.888830 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c69bd-6912-454e-a3ab-0b89ee2e4fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:42.889536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c69bd-6912-454e-a3ab-0b89ee2e4fbe None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:42.889831 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c69bd-6912-454e-a3ab-0b89ee2e4fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:42.890104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c69bd-6912-454e-a3ab-0b89ee2e4fbe None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:42.890867 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c69bd-6912-454e-a3ab-0b89ee2e4fbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:42.891176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c69bd-6912-454e-a3ab-0b89ee2e4fbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:42.892544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c69bd-6912-454e-a3ab-0b89ee2e4fbe None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:42.901007 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 518/1055] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:03:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:42.907320 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cce94-0837-4ee6-945d-c4158cd9bb24 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:42.986387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:43.002372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1a5f0-caff-4d7b-9fdb-b7868fd5a7a9 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:43.002665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1a5f0-caff-4d7b-9fdb-b7868fd5a7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:43.002945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1a5f0-caff-4d7b-9fdb-b7868fd5a7a9 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:43.004358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 519/1056] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:03:43.009386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32c2cc-b79a-4400-a3b2-c40d103ac89b None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:43.009736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32c2cc-b79a-4400-a3b2-c40d103ac89b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:43.009957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32c2cc-b79a-4400-a3b2-c40d103ac89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:43.026801 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cce94-0837-4ee6-945d-c4158cd9bb24 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:43.029315 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-ca32c2cc-b79a-4400-a3b2-c40d103ac89b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:43.030509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ca32c2cc-b79a-4400-a3b2-c40d103ac89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:43.061740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f6d27-65a6-40a8-8c60-5aa70fa9530d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:43.062611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f6d27-65a6-40a8-8c60-5aa70fa9530d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:43.062891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f6d27-65a6-40a8-8c60-5aa70fa9530d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:43.063165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f6d27-65a6-40a8-8c60-5aa70fa9530d None admin] PATH_INFO: `/v3/users/d5d4f98c4aeb404bbcf31965b6ee8e61` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:43.077396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f6d27-65a6-40a8-8c60-5aa70fa9530d None admin] RBAC: Authorizing `identity:get_user(user_id=d5d4f98c4aeb404bbcf31965b6ee8e61)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:43.078855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f6d27-65a6-40a8-8c60-5aa70fa9530d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:43.080561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 538/1057] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:03:42 2022] GET /identity/v3/users/d5d4f98c4aeb404bbcf31965b6ee8e61 => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:43.081813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d42a8-6b53-41aa-a496-0c0f1259e2db None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:43.082105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d42a8-6b53-41aa-a496-0c0f1259e2db None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:43.082386 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d42a8-6b53-41aa-a496-0c0f1259e2db None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:43.083623 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 539/1058] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:43.086147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d42a8-6b53-41aa-a496-0c0f1259e2db None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:43.101406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 520/1059] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:43.102687 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d466e3-aa33-41c8-82b1-bf0afa76462a None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:43.102977 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d466e3-aa33-41c8-82b1-bf0afa76462a None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:43.103255 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d466e3-aa33-41c8-82b1-bf0afa76462a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:43.121729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b5d466e3-aa33-41c8-82b1-bf0afa76462a None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:43.123008 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b5d466e3-aa33-41c8-82b1-bf0afa76462a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:43.190134 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 521/1060] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:43.190983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d466e3-aa33-41c8-82b1-bf0afa76462a None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:43.193578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d42a8-6b53-41aa-a496-0c0f1259e2db None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:43.212483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:43.220217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:43.231373 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692db96e-447f-4f4a-8d99-76e713eaef34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:43.232025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692db96e-447f-4f4a-8d99-76e713eaef34 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:43.232310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692db96e-447f-4f4a-8d99-76e713eaef34 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:43.232586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692db96e-447f-4f4a-8d99-76e713eaef34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:43.292821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d466e3-aa33-41c8-82b1-bf0afa76462a None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:43.301869 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692db96e-447f-4f4a-8d99-76e713eaef34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:43.303326 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692db96e-447f-4f4a-8d99-76e713eaef34 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:43.327678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9722dd-4a8f-4995-9121-04c31ff96fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:43.328494 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9722dd-4a8f-4995-9121-04c31ff96fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:43.328785 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9722dd-4a8f-4995-9121-04c31ff96fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:43.329098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9722dd-4a8f-4995-9121-04c31ff96fc7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:43.339404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9722dd-4a8f-4995-9121-04c31ff96fc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-5f9722dd-4a8f-4995-9121-04c31ff96fc7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:03:43.347853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:43.356303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:43.357330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:43.357330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:43.357330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:43.357330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:43.357330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:03:43.357330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:03:43.357330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:03:43.357330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:03:43.357330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:03:43.357330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:03:43.357330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:03:43.357330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:43.357330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 522/1061] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Jun 15 10:03:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:43.357330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9722dd-4a8f-4995-9121-04c31ff96fc7 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:43.410479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 540/1062] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:03:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 10:03:43.411414 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692db96e-447f-4f4a-8d99-76e713eaef34 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:43.435455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:43.437304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:43.456616 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9722dd-4a8f-4995-9121-04c31ff96fc7 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:43.490473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32768d78-5d29-49f5-98d2-0d3e23cfc89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:43.491122 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32768d78-5d29-49f5-98d2-0d3e23cfc89d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:43.491415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32768d78-5d29-49f5-98d2-0d3e23cfc89d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:43.491699 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32768d78-5d29-49f5-98d2-0d3e23cfc89d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:43.562016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692db96e-447f-4f4a-8d99-76e713eaef34 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:43.594763 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32768d78-5d29-49f5-98d2-0d3e23cfc89d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:43.596238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32768d78-5d29-49f5-98d2-0d3e23cfc89d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:43.596792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf84efd-a247-4f19-a698-dce96e550bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:43.597486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf84efd-a247-4f19-a698-dce96e550bdc None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:43.597766 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf84efd-a247-4f19-a698-dce96e550bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:43.598040 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf84efd-a247-4f19-a698-dce96e550bdc None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:43.598789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf84efd-a247-4f19-a698-dce96e550bdc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:43.599091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf84efd-a247-4f19-a698-dce96e550bdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:43.600441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf84efd-a247-4f19-a698-dce96e550bdc None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:43.609171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 541/1063] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Jun 15 10:03:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:43.609972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf84efd-a247-4f19-a698-dce96e550bdc None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:43.677911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:43.697595 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 523/1064] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:03:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:03:43.698251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:43.698397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32768d78-5d29-49f5-98d2-0d3e23cfc89d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:43.718562 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf84efd-a247-4f19-a698-dce96e550bdc None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:43.752874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9422c-9604-41bc-a72a-3ddb318124b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:43.753767 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9422c-9604-41bc-a72a-3ddb318124b0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:43.754059 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9422c-9604-41bc-a72a-3ddb318124b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:43.754338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9422c-9604-41bc-a72a-3ddb318124b0 None admin] PATH_INFO: `/v3/projects/e5db78f6a088405d997cb70550dc091e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:43.755509 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9422c-9604-41bc-a72a-3ddb318124b0 None admin] RBAC: Authorizing `identity:get_project(project_id=e5db78f6a088405d997cb70550dc091e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:43.756980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9422c-9604-41bc-a72a-3ddb318124b0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:43.758683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 542/1065] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:03:43 2022] GET /identity/v3/projects/e5db78f6a088405d997cb70550dc091e => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:43.759437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9422c-9604-41bc-a72a-3ddb318124b0 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:43.802969 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32768d78-5d29-49f5-98d2-0d3e23cfc89d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:43.838539 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdcca4-88fa-4208-b0ea-ae263d88a01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:43.839851 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdcca4-88fa-4208-b0ea-ae263d88a01d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:43.840351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdcca4-88fa-4208-b0ea-ae263d88a01d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:43.840636 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccdcca4-88fa-4208-b0ea-ae263d88a01d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:43.851117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccdcca4-88fa-4208-b0ea-ae263d88a01d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:43.852559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccdcca4-88fa-4208-b0ea-ae263d88a01d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:43.859978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9422c-9604-41bc-a72a-3ddb318124b0 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:43.871205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-fccdcca4-88fa-4208-b0ea-ae263d88a01d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:43.871205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:43.871205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:43.871205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:43.871205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:43.871205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:43.871205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:43.871205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:43.871205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:43.871205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:43.871205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:43.871205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:43.871205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:43.871205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:43.871205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:43.872114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:43.873041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:43.873041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:43.873041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:43.873041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:43.873041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 524/1066] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:43.873626 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdcca4-88fa-4208-b0ea-ae263d88a01d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:43.901510 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324efc7a-16ea-40d5-92fa-aad59b80d733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:43.903117 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324efc7a-16ea-40d5-92fa-aad59b80d733 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:43.903434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324efc7a-16ea-40d5-92fa-aad59b80d733 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:43.903791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324efc7a-16ea-40d5-92fa-aad59b80d733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:43.904648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:43.917676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324efc7a-16ea-40d5-92fa-aad59b80d733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:43.919496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324efc7a-16ea-40d5-92fa-aad59b80d733 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:43.938661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-324efc7a-16ea-40d5-92fa-aad59b80d733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:43.938661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:43.938661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:43.938661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:43.938661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:43.938661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:43.938661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:43.938661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:43.938661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:43.938661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:43.938661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:43.938661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:43.938661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:43.938661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:43.938661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:43.939813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:43.940920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:43.940920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:43.940920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:43.940920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:43.940920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 543/1067] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:43.945548 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324efc7a-16ea-40d5-92fa-aad59b80d733 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:43.955800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:43.976751 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccdcca4-88fa-4208-b0ea-ae263d88a01d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:44.013604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d42c7-5722-481e-87cf-a67d8fb42ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:44.014260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d42c7-5722-481e-87cf-a67d8fb42ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:44.014541 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d42c7-5722-481e-87cf-a67d8fb42ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:44.014813 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d42c7-5722-481e-87cf-a67d8fb42ae9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:44.015652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d42c7-5722-481e-87cf-a67d8fb42ae9 None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=e5db78f6a088405d997cb70550dc091e, user.id=d5d4f98c4aeb404bbcf31965b6ee8e61) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:44.015963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d42c7-5722-481e-87cf-a67d8fb42ae9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:44.017427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d42c7-5722-481e-87cf-a67d8fb42ae9 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:44.023000 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 525/1068] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:03:43 2022] GET /identity/v3/role_assignments?user.id=d5d4f98c4aeb404bbcf31965b6ee8e61&scope.project.id=e5db78f6a088405d997cb70550dc091e&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:44.023791 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d42c7-5722-481e-87cf-a67d8fb42ae9 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:44.071997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324efc7a-16ea-40d5-92fa-aad59b80d733 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:44.122011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802c73b-a5b8-4855-a9c6-6a6fb43cc993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:44.123172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802c73b-a5b8-4855-a9c6-6a6fb43cc993 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:44.123768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802c73b-a5b8-4855-a9c6-6a6fb43cc993 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:44.124353 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802c73b-a5b8-4855-a9c6-6a6fb43cc993 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:44.125442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802c73b-a5b8-4855-a9c6-6a6fb43cc993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:44.125992 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802c73b-a5b8-4855-a9c6-6a6fb43cc993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:44.127734 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802c73b-a5b8-4855-a9c6-6a6fb43cc993 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:44.133986 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:44.141329 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d42c7-5722-481e-87cf-a67d8fb42ae9 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:44.145205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 544/1069] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:44.151534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802c73b-a5b8-4855-a9c6-6a6fb43cc993 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:44.181973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028c68e-0526-450c-8bc3-8d5c29987dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:44.182647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028c68e-0526-450c-8bc3-8d5c29987dac None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:44.183263 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028c68e-0526-450c-8bc3-8d5c29987dac None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:44.183263 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028c68e-0526-450c-8bc3-8d5c29987dac None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:44.184310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028c68e-0526-450c-8bc3-8d5c29987dac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:44.184620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028c68e-0526-450c-8bc3-8d5c29987dac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:44.186019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028c68e-0526-450c-8bc3-8d5c29987dac None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:44.199193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 526/1070] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:44.225946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:44.266247 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802c73b-a5b8-4855-a9c6-6a6fb43cc993 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:44.304731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0f475-0e60-4a70-a1f6-06441f640a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:44.305587 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0f475-0e60-4a70-a1f6-06441f640a57 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:44.305874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0f475-0e60-4a70-a1f6-06441f640a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:44.306152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0f475-0e60-4a70-a1f6-06441f640a57 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:44.314942 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0f475-0e60-4a70-a1f6-06441f640a57 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:44.316517 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0f475-0e60-4a70-a1f6-06441f640a57 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-f9d0f475-0e60-4a70-a1f6-06441f640a57 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:03:44.333185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:44.334232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:44.335175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:44.335175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:44.335175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:03:44.335175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:03:44.335175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:03:44.335175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:03:44.335175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:03:44.335175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:03:44.335175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:03:44.335175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:44.335175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 545/1071] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jun 15 10:03:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:44.340910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028c68e-0526-450c-8bc3-8d5c29987dac None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:44.448110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028c68e-0526-450c-8bc3-8d5c29987dac None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:44.487442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b950022-5182-4b3a-bfe6-f888fd4d8cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:44.488843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b950022-5182-4b3a-bfe6-f888fd4d8cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:44.489367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b950022-5182-4b3a-bfe6-f888fd4d8cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:44.489806 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b950022-5182-4b3a-bfe6-f888fd4d8cf5 None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:44.490669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b950022-5182-4b3a-bfe6-f888fd4d8cf5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:44.490984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b950022-5182-4b3a-bfe6-f888fd4d8cf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:44.492416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b950022-5182-4b3a-bfe6-f888fd4d8cf5 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:44.511648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 527/1072] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jun 15 10:03:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:44.517723 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0f475-0e60-4a70-a1f6-06441f640a57 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:44.625878 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0f475-0e60-4a70-a1f6-06441f640a57 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:44.653155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:44.661895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea150a-49f0-47f2-b6a8-259c9de46160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:44.662690 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea150a-49f0-47f2-b6a8-259c9de46160 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:44.662974 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea150a-49f0-47f2-b6a8-259c9de46160 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:44.663252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea150a-49f0-47f2-b6a8-259c9de46160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:44.673852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea150a-49f0-47f2-b6a8-259c9de46160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:44.675638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea150a-49f0-47f2-b6a8-259c9de46160 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:44.693265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-a8ea150a-49f0-47f2-b6a8-259c9de46160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:44.693265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:44.693265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:44.693265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:44.693265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:44.693265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:44.693265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:44.693265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:44.693265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:44.693265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:44.693265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:44.693265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:44.693265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:44.693265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:44.693265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:44.694330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:44.700535 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:44.700535 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:44.700535 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:44.700535 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:44.700535 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 546/1073] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:44.701956 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b950022-5182-4b3a-bfe6-f888fd4d8cf5 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:44.718270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:44.804690 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b950022-5182-4b3a-bfe6-f888fd4d8cf5 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:44.838525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91251447-ffbe-43b5-bc6a-bf9efe99173a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:44.839220 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91251447-ffbe-43b5-bc6a-bf9efe99173a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:44.839505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91251447-ffbe-43b5-bc6a-bf9efe99173a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:44.839781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91251447-ffbe-43b5-bc6a-bf9efe99173a None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:44.840515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91251447-ffbe-43b5-bc6a-bf9efe99173a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:44.840822 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91251447-ffbe-43b5-bc6a-bf9efe99173a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:44.842222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91251447-ffbe-43b5-bc6a-bf9efe99173a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:44.853470 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 528/1074] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:44.859740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea150a-49f0-47f2-b6a8-259c9de46160 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:44.962109 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea150a-49f0-47f2-b6a8-259c9de46160 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:44.996984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927aecc-be9a-4da8-9459-efd8d72b212b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:44.997810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927aecc-be9a-4da8-9459-efd8d72b212b None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:44.998100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927aecc-be9a-4da8-9459-efd8d72b212b None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:44.998383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927aecc-be9a-4da8-9459-efd8d72b212b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:45.005181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927aecc-be9a-4da8-9459-efd8d72b212b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:45.006635 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1927aecc-be9a-4da8-9459-efd8d72b212b None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:45.020294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-1927aecc-be9a-4da8-9459-efd8d72b212b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:45.020294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:45.020294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:45.020294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:45.020294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:45.020294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:45.020294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:45.020294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:45.020294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:45.020294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:45.020294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:45.020294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:45.020294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:03:45.020294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:03:45.020294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:45.021255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:45.022141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:03:45.022141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:03:45.022141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:03:45.022141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:03:45.022141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:03:45.022141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:03:45.022141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:45.022141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:45.022141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 547/1075] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:45.027794 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91251447-ffbe-43b5-bc6a-bf9efe99173a None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:45.102606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:45.130803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91251447-ffbe-43b5-bc6a-bf9efe99173a None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:45.164361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b00b22-3b22-4132-a155-e900aba4d9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:45.165057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b00b22-3b22-4132-a155-e900aba4d9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:45.165341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b00b22-3b22-4132-a155-e900aba4d9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:45.165620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b00b22-3b22-4132-a155-e900aba4d9d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:45.166417 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b00b22-3b22-4132-a155-e900aba4d9d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:45.166730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b00b22-3b22-4132-a155-e900aba4d9d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:45.168186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b00b22-3b22-4132-a155-e900aba4d9d8 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:45.180769 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 529/1076] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:03:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:45.187292 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927aecc-be9a-4da8-9459-efd8d72b212b None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:45.230890 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:45.299804 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927aecc-be9a-4da8-9459-efd8d72b212b None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:45.334146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6efd6-eff4-4f75-8426-b7f8ef6e5162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:45.335037 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6efd6-eff4-4f75-8426-b7f8ef6e5162 None admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:45.335325 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6efd6-eff4-4f75-8426-b7f8ef6e5162 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:45.335605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6efd6-eff4-4f75-8426-b7f8ef6e5162 None admin] PATH_INFO: `/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/26f23f8595fc43ad94aee7d0192164a1/roles/b1c676dcdeaa4ac0a2a3a7ef96317fa0` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:45.337858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6efd6-eff4-4f75-8426-b7f8ef6e5162 None admin] RBAC: Authorizing `identity:create_grant(project_id=db47f1bb607c4d1a9c346fdc80ecadfa, user_id=26f23f8595fc43ad94aee7d0192164a1, role_id=b1c676dcdeaa4ac0a2a3a7ef96317fa0)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:45.339928 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6efd6-eff4-4f75-8426-b7f8ef6e5162 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:45.415505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 548/1077] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:03:45 2022] PUT /identity/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/26f23f8595fc43ad94aee7d0192164a1/roles/b1c676dcdeaa4ac0a2a3a7ef96317fa0 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:03:46.201339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64390e5-8cc9-46fb-832f-fc7029a0424d None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:46.201870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64390e5-8cc9-46fb-832f-fc7029a0424d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:46.201870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64390e5-8cc9-46fb-832f-fc7029a0424d None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:46.203291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 530/1078] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:46.208850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295fb6f-2b38-44d8-9c75-5e0f78cf97bd None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:46.209360 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295fb6f-2b38-44d8-9c75-5e0f78cf97bd None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:46.209578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295fb6f-2b38-44d8-9c75-5e0f78cf97bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:46.226514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-b295fb6f-2b38-44d8-9c75-5e0f78cf97bd None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:46.227698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b295fb6f-2b38-44d8-9c75-5e0f78cf97bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:46.294106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 549/1079] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:46.317564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64390e5-8cc9-46fb-832f-fc7029a0424d None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:46.417882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64390e5-8cc9-46fb-832f-fc7029a0424d None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:46.452807 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ac8b0-828b-4d5c-94ce-2891e4a08995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:46.453563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ac8b0-828b-4d5c-94ce-2891e4a08995 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:46.453844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ac8b0-828b-4d5c-94ce-2891e4a08995 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:46.454118 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ac8b0-828b-4d5c-94ce-2891e4a08995 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:46.456111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:46.465740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ac8b0-828b-4d5c-94ce-2891e4a08995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-365ac8b0-828b-4d5c-94ce-2891e4a08995 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:03:46.473592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:46.478424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:46.479414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:46.479414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:46.479414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:46.479414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:46.479414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:03:46.479414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:03:46.479414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:03:46.479414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:03:46.479414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:03:46.479414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:03:46.479414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:03:46.479414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:46.479414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 531/1080] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:03:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:46.481905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295fb6f-2b38-44d8-9c75-5e0f78cf97bd None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:46.512608 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:46.586101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295fb6f-2b38-44d8-9c75-5e0f78cf97bd None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:46.620289 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac6c32-5c94-454c-ad5f-bf65a2b1fbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:46.620976 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac6c32-5c94-454c-ad5f-bf65a2b1fbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:46.621258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac6c32-5c94-454c-ad5f-bf65a2b1fbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:46.621534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac6c32-5c94-454c-ad5f-bf65a2b1fbd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:46.622307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac6c32-5c94-454c-ad5f-bf65a2b1fbd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:46.622611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac6c32-5c94-454c-ad5f-bf65a2b1fbd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:46.623975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac6c32-5c94-454c-ad5f-bf65a2b1fbd3 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:46.632348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 550/1081] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:03:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:46.638957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ac8b0-828b-4d5c-94ce-2891e4a08995 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:46.738112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ac8b0-828b-4d5c-94ce-2891e4a08995 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:46.771412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab612387-26eb-4f12-b1f5-8a1dc300d70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:46.772281 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab612387-26eb-4f12-b1f5-8a1dc300d70d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:46.772558 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab612387-26eb-4f12-b1f5-8a1dc300d70d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:46.772827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab612387-26eb-4f12-b1f5-8a1dc300d70d None admin] PATH_INFO: `/v3/users/d5d4f98c4aeb404bbcf31965b6ee8e61` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:46.774025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab612387-26eb-4f12-b1f5-8a1dc300d70d None admin] RBAC: Authorizing `identity:get_user(user_id=d5d4f98c4aeb404bbcf31965b6ee8e61)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:46.775445 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab612387-26eb-4f12-b1f5-8a1dc300d70d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:46.781879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 532/1082] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:03:46 2022] GET /identity/v3/users/d5d4f98c4aeb404bbcf31965b6ee8e61 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:46.787566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac6c32-5c94-454c-ad5f-bf65a2b1fbd3 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:46.879788 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:46.890426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac6c32-5c94-454c-ad5f-bf65a2b1fbd3 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:46.925134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf871cd-6f47-4eda-ad79-d7959f9c4e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:46.925998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf871cd-6f47-4eda-ad79-d7959f9c4e76 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:46.926276 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf871cd-6f47-4eda-ad79-d7959f9c4e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:46.926549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf871cd-6f47-4eda-ad79-d7959f9c4e76 None admin] PATH_INFO: `/v3/projects/e5db78f6a088405d997cb70550dc091e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:46.927715 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf871cd-6f47-4eda-ad79-d7959f9c4e76 None admin] RBAC: Authorizing `identity:get_project(project_id=e5db78f6a088405d997cb70550dc091e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:46.929171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf871cd-6f47-4eda-ad79-d7959f9c4e76 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:46.930875 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 551/1083] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:03:46 2022] GET /identity/v3/projects/e5db78f6a088405d997cb70550dc091e => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:46.936867 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab612387-26eb-4f12-b1f5-8a1dc300d70d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:46.998310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:47.038269 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab612387-26eb-4f12-b1f5-8a1dc300d70d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:47.072989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27bf85-c77d-4bbf-a5cc-a541e1f082b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:47.074436 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27bf85-c77d-4bbf-a5cc-a541e1f082b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:47.074767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27bf85-c77d-4bbf-a5cc-a541e1f082b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:47.074987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27bf85-c77d-4bbf-a5cc-a541e1f082b4 None admin] PATH_INFO: `/v3/projects/e5db78f6a088405d997cb70550dc091e/users/d5d4f98c4aeb404bbcf31965b6ee8e61/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:47.077149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27bf85-c77d-4bbf-a5cc-a541e1f082b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5db78f6a088405d997cb70550dc091e, user_id=d5d4f98c4aeb404bbcf31965b6ee8e61, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:47.078592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27bf85-c77d-4bbf-a5cc-a541e1f082b4 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:47.145748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 533/1084] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:03:46 2022] PUT /identity/v3/projects/e5db78f6a088405d997cb70550dc091e/users/d5d4f98c4aeb404bbcf31965b6ee8e61/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:03:47.564600 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29151b75-31ab-4216-b395-2eacc8aa2d2a None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:47.565170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29151b75-31ab-4216-b395-2eacc8aa2d2a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:47.565170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29151b75-31ab-4216-b395-2eacc8aa2d2a None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:47.566634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 552/1085] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:47.572506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e8a64-c0a8-4690-92f5-b3321e2cc3bc None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:47.573031 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e8a64-c0a8-4690-92f5-b3321e2cc3bc None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:47.573125 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e8a64-c0a8-4690-92f5-b3321e2cc3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:47.590364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-f97e8a64-c0a8-4690-92f5-b3321e2cc3bc None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:47.591416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f97e8a64-c0a8-4690-92f5-b3321e2cc3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:47.665751 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 534/1086] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:47.689746 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29151b75-31ab-4216-b395-2eacc8aa2d2a None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:47.794338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29151b75-31ab-4216-b395-2eacc8aa2d2a None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:47.828740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87ee34-1aae-42f8-a873-c1f021457a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:47.829605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87ee34-1aae-42f8-a873-c1f021457a7c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:47.829894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87ee34-1aae-42f8-a873-c1f021457a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:47.830170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87ee34-1aae-42f8-a873-c1f021457a7c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:47.841952 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87ee34-1aae-42f8-a873-c1f021457a7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-7b87ee34-1aae-42f8-a873-c1f021457a7c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:03:47.850288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:47.851479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:47.857755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:47.857755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:47.857755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:47.857755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:47.857755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:03:47.857755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:03:47.857755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:03:47.857755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:03:47.857755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:03:47.857755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:03:47.857755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 15 10:03:47.857755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:47.857755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 553/1087] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Jun 15 10:03:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:47.858513 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e8a64-c0a8-4690-92f5-b3321e2cc3bc None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:47.910697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:47.974088 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e8a64-c0a8-4690-92f5-b3321e2cc3bc None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:48.011153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9738d2-29a8-4a5d-8feb-1fc486d4f299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:48.011812 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9738d2-29a8-4a5d-8feb-1fc486d4f299 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:48.012093 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9738d2-29a8-4a5d-8feb-1fc486d4f299 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:48.012366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9738d2-29a8-4a5d-8feb-1fc486d4f299 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:48.013165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9738d2-29a8-4a5d-8feb-1fc486d4f299 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:48.013475 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9738d2-29a8-4a5d-8feb-1fc486d4f299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:48.014845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9738d2-29a8-4a5d-8feb-1fc486d4f299 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:48.024732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 535/1088] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Jun 15 10:03:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:48.031838 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87ee34-1aae-42f8-a873-c1f021457a7c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:48.052267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:48.136443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87ee34-1aae-42f8-a873-c1f021457a7c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:48.172751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466918d-b26f-4691-b2cf-19765d61018b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:48.173589 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466918d-b26f-4691-b2cf-19765d61018b None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:48.173877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466918d-b26f-4691-b2cf-19765d61018b None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:48.174153 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466918d-b26f-4691-b2cf-19765d61018b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:48.184819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466918d-b26f-4691-b2cf-19765d61018b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:48.186312 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0466918d-b26f-4691-b2cf-19765d61018b None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:48.204121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-0466918d-b26f-4691-b2cf-19765d61018b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:48.204121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:48.204121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:48.204121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:48.204121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:48.204121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:48.204121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:48.204121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:48.204121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:48.204121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:48.204121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:48.204121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:48.204121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:48.204121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:48.204121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:48.205228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:48.206171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:48.206171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:48.206171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:48.206171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:48.206171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 554/1089] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:48.211433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9738d2-29a8-4a5d-8feb-1fc486d4f299 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:48.315461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9738d2-29a8-4a5d-8feb-1fc486d4f299 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:48.329831 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:48.350398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bce6e-93c1-445b-9dc6-daec4c1e5f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:48.351089 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bce6e-93c1-445b-9dc6-daec4c1e5f71 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:48.351357 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bce6e-93c1-445b-9dc6-daec4c1e5f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:48.351633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bce6e-93c1-445b-9dc6-daec4c1e5f71 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:48.352365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9bce6e-93c1-445b-9dc6-daec4c1e5f71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:48.352675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9bce6e-93c1-445b-9dc6-daec4c1e5f71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:48.354097 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9bce6e-93c1-445b-9dc6-daec4c1e5f71 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:48.367632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 536/1090] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:48.373171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466918d-b26f-4691-b2cf-19765d61018b None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:48.448158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:48.485003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466918d-b26f-4691-b2cf-19765d61018b None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:48.519937 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2b068-79c1-49eb-b145-c81d0c866545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:48.520739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2b068-79c1-49eb-b145-c81d0c866545 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:48.521084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2b068-79c1-49eb-b145-c81d0c866545 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:48.521368 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2b068-79c1-49eb-b145-c81d0c866545 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:48.529962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2b068-79c1-49eb-b145-c81d0c866545 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:48.531424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2b068-79c1-49eb-b145-c81d0c866545 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-0fb2b068-79c1-49eb-b145-c81d0c866545 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:03:48.545971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:48.547099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:48.548058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:48.548058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:48.548058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:03:48.548058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:03:48.548058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:03:48.548058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:03:48.548058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:03:48.548058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:03:48.548058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 15 10:03:48.548058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:48.548058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 555/1091] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jun 15 10:03:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:48.553549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bce6e-93c1-445b-9dc6-daec4c1e5f71 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:48.654865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bce6e-93c1-445b-9dc6-daec4c1e5f71 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:48.690317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb85d75-81b0-4ba0-80fe-3d539aaee229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:48.691017 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb85d75-81b0-4ba0-80fe-3d539aaee229 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:48.691302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb85d75-81b0-4ba0-80fe-3d539aaee229 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:48.691581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb85d75-81b0-4ba0-80fe-3d539aaee229 None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:48.692435 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb85d75-81b0-4ba0-80fe-3d539aaee229 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:48.692748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb85d75-81b0-4ba0-80fe-3d539aaee229 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:48.694230 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb85d75-81b0-4ba0-80fe-3d539aaee229 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:48.713583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 537/1092] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jun 15 10:03:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:48.719724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2b068-79c1-49eb-b145-c81d0c866545 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:48.822170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2b068-79c1-49eb-b145-c81d0c866545 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:48.851446 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:48.858147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cbe22-62ee-470a-b0ac-3b85e944d434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:48.858816 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cbe22-62ee-470a-b0ac-3b85e944d434 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:48.859101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cbe22-62ee-470a-b0ac-3b85e944d434 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:48.859378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95cbe22-62ee-470a-b0ac-3b85e944d434 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:48.929181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95cbe22-62ee-470a-b0ac-3b85e944d434 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:48.930553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95cbe22-62ee-470a-b0ac-3b85e944d434 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:49.032022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 556/1093] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:03:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:03:49.037573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb85d75-81b0-4ba0-80fe-3d539aaee229 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:49.069757 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:49.142074 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb85d75-81b0-4ba0-80fe-3d539aaee229 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:49.177905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101caf8c-d221-4609-b3a0-752a24d1314a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:49.178704 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101caf8c-d221-4609-b3a0-752a24d1314a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:49.178987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101caf8c-d221-4609-b3a0-752a24d1314a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:49.179311 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101caf8c-d221-4609-b3a0-752a24d1314a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:49.190457 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101caf8c-d221-4609-b3a0-752a24d1314a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:49.191916 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101caf8c-d221-4609-b3a0-752a24d1314a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:49.209707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-101caf8c-d221-4609-b3a0-752a24d1314a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:49.209707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:49.209707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:49.209707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:49.209707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:49.209707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:49.209707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:49.209707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:49.209707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:49.209707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:49.209707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:49.209707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:49.209707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:03:49.209707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:03:49.209707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:49.210821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:49.211803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:03:49.211803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:03:49.211803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:03:49.211803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:49.211803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 538/1094] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:03:49.217151 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cbe22-62ee-470a-b0ac-3b85e944d434 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:49.249035 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:49.323416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95cbe22-62ee-470a-b0ac-3b85e944d434 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:49.326153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e00968-e515-4ed4-b02f-efac61b256d0 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:49.326448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e00968-e515-4ed4-b02f-efac61b256d0 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:49.326755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e00968-e515-4ed4-b02f-efac61b256d0 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:49.328114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 539/1095] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:03:49.333327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4f695-2d91-4d77-8cde-648417505cff None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:49.333611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4f695-2d91-4d77-8cde-648417505cff None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:49.333887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4f695-2d91-4d77-8cde-648417505cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:49.352011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-1da4f695-2d91-4d77-8cde-648417505cff None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:49.353250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1da4f695-2d91-4d77-8cde-648417505cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:49.357855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e93e9-716d-4687-b4b8-41c1c3842763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:49.358522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e93e9-716d-4687-b4b8-41c1c3842763 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:49.358808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e93e9-716d-4687-b4b8-41c1c3842763 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:49.359083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e93e9-716d-4687-b4b8-41c1c3842763 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:49.359816 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e93e9-716d-4687-b4b8-41c1c3842763 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:49.360124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e93e9-716d-4687-b4b8-41c1c3842763 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:49.361538 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e93e9-716d-4687-b4b8-41c1c3842763 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:49.373743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 557/1096] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:49.379457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e93e9-716d-4687-b4b8-41c1c3842763 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:49.423546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 540/1097] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:03:49.447627 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4f695-2d91-4d77-8cde-648417505cff None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:49.463652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:49.470999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:49.480433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e93e9-716d-4687-b4b8-41c1c3842763 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:49.514269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd44fc7-7db9-4c2a-a4aa-eac51d026824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:49.515076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd44fc7-7db9-4c2a-a4aa-eac51d026824 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:49.515364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd44fc7-7db9-4c2a-a4aa-eac51d026824 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:49.515646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd44fc7-7db9-4c2a-a4aa-eac51d026824 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:49.522708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd44fc7-7db9-4c2a-a4aa-eac51d026824 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:49.524794 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd44fc7-7db9-4c2a-a4aa-eac51d026824 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:49.539722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-2cd44fc7-7db9-4c2a-a4aa-eac51d026824 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:49.539722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:49.539722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:49.539722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:49.539722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:49.539722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:49.539722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:49.539722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:49.539722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:49.539722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:49.539722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:49.539722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:49.539722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:03:49.539722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:03:49.539722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:49.540665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:49.542113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:03:49.542113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:03:49.542113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:03:49.542113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:03:49.542113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:03:49.542113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:03:49.542113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:03:49.542113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:03:49.542113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 558/1098] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:03:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:49.546842 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd44fc7-7db9-4c2a-a4aa-eac51d026824 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:49.554616 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4f695-2d91-4d77-8cde-648417505cff None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:49.588930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03879ab-7890-483f-89c5-43b458df0f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:49.589827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03879ab-7890-483f-89c5-43b458df0f14 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:49.590109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03879ab-7890-483f-89c5-43b458df0f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:49.590384 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03879ab-7890-483f-89c5-43b458df0f14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:49.600450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03879ab-7890-483f-89c5-43b458df0f14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-e03879ab-7890-483f-89c5-43b458df0f14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:03:49.608227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:03:49.609253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:03:49.610107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:03:49.610107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:03:49.610107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:03:49.610107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:03:49.610107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:03:49.610107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:03:49.610107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:03:49.610107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:03:49.610107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:03:49.610107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:03:49.610107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:03:49.610107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:03:49.610107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 541/1099] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:03:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:03:49.615351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03879ab-7890-483f-89c5-43b458df0f14 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:49.650111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd44fc7-7db9-4c2a-a4aa-eac51d026824 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:49.685945 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff05970-ad6d-4dce-b232-19212d7792b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:49.686638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff05970-ad6d-4dce-b232-19212d7792b1 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:49.686922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff05970-ad6d-4dce-b232-19212d7792b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:49.687204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff05970-ad6d-4dce-b232-19212d7792b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:49.688012 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff05970-ad6d-4dce-b232-19212d7792b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:49.688324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff05970-ad6d-4dce-b232-19212d7792b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:49.689795 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff05970-ad6d-4dce-b232-19212d7792b1 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:49.692719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:49.704127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 559/1100] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:03:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:49.707614 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff05970-ad6d-4dce-b232-19212d7792b1 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:49.718630 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03879ab-7890-483f-89c5-43b458df0f14 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:49.721135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:49.756520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5336ad-e884-4fae-adfe-c008f35f4f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:49.757228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5336ad-e884-4fae-adfe-c008f35f4f3a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:49.757510 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5336ad-e884-4fae-adfe-c008f35f4f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:49.757782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5336ad-e884-4fae-adfe-c008f35f4f3a None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:49.758551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5336ad-e884-4fae-adfe-c008f35f4f3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:49.758858 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5336ad-e884-4fae-adfe-c008f35f4f3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:49.760247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5336ad-e884-4fae-adfe-c008f35f4f3a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:49.769326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 542/1101] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:03:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:49.775246 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5336ad-e884-4fae-adfe-c008f35f4f3a None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:49.815768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff05970-ad6d-4dce-b232-19212d7792b1 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:49.848816 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbb536-ac2b-4758-b92a-7607e23d23b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:49.849494 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbb536-ac2b-4758-b92a-7607e23d23b4 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:49.849776 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbb536-ac2b-4758-b92a-7607e23d23b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:49.850051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbb536-ac2b-4758-b92a-7607e23d23b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:49.850892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbb536-ac2b-4758-b92a-7607e23d23b4 None admin] RBAC: Adding query filter params (role.id=b1c676dcdeaa4ac0a2a3a7ef96317fa0, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=26f23f8595fc43ad94aee7d0192164a1) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:49.851202 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbb536-ac2b-4758-b92a-7607e23d23b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:49.852647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbb536-ac2b-4758-b92a-7607e23d23b4 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:49.859768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 560/1102] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:03:49 2022] GET /identity/v3/role_assignments?user.id=26f23f8595fc43ad94aee7d0192164a1&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=b1c676dcdeaa4ac0a2a3a7ef96317fa0 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:49.880944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5336ad-e884-4fae-adfe-c008f35f4f3a None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:49.919575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141d473-30d3-461b-97cb-8873bd7af311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:49.920485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141d473-30d3-461b-97cb-8873bd7af311 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:49.920768 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141d473-30d3-461b-97cb-8873bd7af311 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:49.921077 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141d473-30d3-461b-97cb-8873bd7af311 None admin] PATH_INFO: `/v3/users/d5d4f98c4aeb404bbcf31965b6ee8e61` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:49.922413 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141d473-30d3-461b-97cb-8873bd7af311 None admin] RBAC: Authorizing `identity:get_user(user_id=d5d4f98c4aeb404bbcf31965b6ee8e61)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:49.923897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141d473-30d3-461b-97cb-8873bd7af311 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:49.926080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 543/1103] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:03:49 2022] GET /identity/v3/users/d5d4f98c4aeb404bbcf31965b6ee8e61 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:03:49.932085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbb536-ac2b-4758-b92a-7607e23d23b4 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:50.015161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:50.040743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbb536-ac2b-4758-b92a-7607e23d23b4 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:50.078533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b04858-4309-4cb8-adb9-9f873f0a3ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:50.079185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b04858-4309-4cb8-adb9-9f873f0a3ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:50.079464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b04858-4309-4cb8-adb9-9f873f0a3ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:50.079736 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b04858-4309-4cb8-adb9-9f873f0a3ed7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:50.149508 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b04858-4309-4cb8-adb9-9f873f0a3ed7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:50.150958 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b04858-4309-4cb8-adb9-9f873f0a3ed7 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:50.234311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:50.260438 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 561/1104] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:03:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:03:50.267242 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141d473-30d3-461b-97cb-8873bd7af311 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:50.271875 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:50.375145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141d473-30d3-461b-97cb-8873bd7af311 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:50.411091 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b014f-5ba3-41b6-a500-bcf3ceceec23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:50.411967 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b014f-5ba3-41b6-a500-bcf3ceceec23 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:50.412247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b014f-5ba3-41b6-a500-bcf3ceceec23 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:50.412517 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b014f-5ba3-41b6-a500-bcf3ceceec23 None admin] PATH_INFO: `/v3/projects/e5db78f6a088405d997cb70550dc091e` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:50.413761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b014f-5ba3-41b6-a500-bcf3ceceec23 None admin] RBAC: Authorizing `identity:get_project(project_id=e5db78f6a088405d997cb70550dc091e)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:50.415196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b014f-5ba3-41b6-a500-bcf3ceceec23 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:50.416901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 544/1105] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:03:50 2022] GET /identity/v3/projects/e5db78f6a088405d997cb70550dc091e => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:50.424022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b04858-4309-4cb8-adb9-9f873f0a3ed7 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:50.528762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b04858-4309-4cb8-adb9-9f873f0a3ed7 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:50.566027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80dbfa-c9fc-4389-bf04-c29c846995b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:50.566693 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80dbfa-c9fc-4389-bf04-c29c846995b8 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:50.566990 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80dbfa-c9fc-4389-bf04-c29c846995b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:50.567271 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80dbfa-c9fc-4389-bf04-c29c846995b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:50.568127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80dbfa-c9fc-4389-bf04-c29c846995b8 None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=e5db78f6a088405d997cb70550dc091e, user.id=d5d4f98c4aeb404bbcf31965b6ee8e61) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:50.568444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80dbfa-c9fc-4389-bf04-c29c846995b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:50.570016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80dbfa-c9fc-4389-bf04-c29c846995b8 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:50.576446 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 562/1106] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:03:50 2022] GET /identity/v3/role_assignments?user.id=d5d4f98c4aeb404bbcf31965b6ee8e61&scope.project.id=e5db78f6a088405d997cb70550dc091e&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:52.785715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105def4-8b6c-400f-bf8a-dd297515266b None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:52.786270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105def4-8b6c-400f-bf8a-dd297515266b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:52.786270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105def4-8b6c-400f-bf8a-dd297515266b None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:52.787616 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 545/1107] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:52.792795 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eeab98-951f-4612-93f0-e2e2cb7aa58c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:52.793210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eeab98-951f-4612-93f0-e2e2cb7aa58c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:52.793334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eeab98-951f-4612-93f0-e2e2cb7aa58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:52.810446 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-99eeab98-951f-4612-93f0-e2e2cb7aa58c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:52.811665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-99eeab98-951f-4612-93f0-e2e2cb7aa58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:52.841549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:52.878159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 563/1108] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:52.904124 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105def4-8b6c-400f-bf8a-dd297515266b None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:52.973980 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:53.003112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105def4-8b6c-400f-bf8a-dd297515266b None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:53.036426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e9240-740d-4cba-a412-2f4c75021891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:53.037236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e9240-740d-4cba-a412-2f4c75021891 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:53.037521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e9240-740d-4cba-a412-2f4c75021891 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:53.037797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e9240-740d-4cba-a412-2f4c75021891 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:53.038957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e9240-740d-4cba-a412-2f4c75021891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:53.040393 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e9240-740d-4cba-a412-2f4c75021891 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:53.042096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 546/1109] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:03:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:53.047920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eeab98-951f-4612-93f0-e2e2cb7aa58c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:53.144062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eeab98-951f-4612-93f0-e2e2cb7aa58c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:53.177467 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbe047-f00d-4f76-8e03-dcc75e38edd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:53.178129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbe047-f00d-4f76-8e03-dcc75e38edd1 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:53.178416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbe047-f00d-4f76-8e03-dcc75e38edd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:53.178700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbe047-f00d-4f76-8e03-dcc75e38edd1 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:53.179415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbe047-f00d-4f76-8e03-dcc75e38edd1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:53.180852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cbe047-f00d-4f76-8e03-dcc75e38edd1 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:53.262880 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 564/1110] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:03:53 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:03:55.440428 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f055e3-e5a7-434d-9d5c-44ced789732f None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:55.441039 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f055e3-e5a7-434d-9d5c-44ced789732f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:55.441039 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f055e3-e5a7-434d-9d5c-44ced789732f None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:55.442434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 547/1111] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:55.448739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02daf9-b2b7-4d8d-a78f-e6f548db27f1 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:55.449301 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02daf9-b2b7-4d8d-a78f-e6f548db27f1 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:55.449301 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02daf9-b2b7-4d8d-a78f-e6f548db27f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:55.467415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-1e02daf9-b2b7-4d8d-a78f-e6f548db27f1 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:55.468633 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-1e02daf9-b2b7-4d8d-a78f-e6f548db27f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:55.487426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:55.538854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 565/1112] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:55.560775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f055e3-e5a7-434d-9d5c-44ced789732f None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:55.663945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f055e3-e5a7-434d-9d5c-44ced789732f None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:55.700734 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9eb9af-e0d0-4a23-b718-0b59702992e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:55.701537 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9eb9af-e0d0-4a23-b718-0b59702992e1 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:55.701820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9eb9af-e0d0-4a23-b718-0b59702992e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:55.702100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9eb9af-e0d0-4a23-b718-0b59702992e1 None admin] PATH_INFO: `/v3/roles/d241ebe8f5654f46a5a92be6f6b0cc36` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:55.703638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9eb9af-e0d0-4a23-b718-0b59702992e1 None admin] RBAC: Authorizing `identity:get_role(role_id=d241ebe8f5654f46a5a92be6f6b0cc36)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:55.705059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9eb9af-e0d0-4a23-b718-0b59702992e1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:55.706473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 548/1113] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:03:55 2022] GET /identity/v3/roles/d241ebe8f5654f46a5a92be6f6b0cc36 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:55.712387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02daf9-b2b7-4d8d-a78f-e6f548db27f1 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:55.816294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02daf9-b2b7-4d8d-a78f-e6f548db27f1 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:55.850078 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc60ab1-9dc7-4b1f-a669-a396b56214df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:55.850946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc60ab1-9dc7-4b1f-a669-a396b56214df None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:55.851232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc60ab1-9dc7-4b1f-a669-a396b56214df None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:55.851508 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc60ab1-9dc7-4b1f-a669-a396b56214df None admin] PATH_INFO: `/v3/users/6241e3a9458f4d629c335692f05fb181` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:55.865428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc60ab1-9dc7-4b1f-a669-a396b56214df None admin] RBAC: Authorizing `identity:get_user(user_id=6241e3a9458f4d629c335692f05fb181)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:55.866899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc60ab1-9dc7-4b1f-a669-a396b56214df None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:55.868630 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 566/1114] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:03:55 2022] GET /identity/v3/users/6241e3a9458f4d629c335692f05fb181 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:55.874446 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9eb9af-e0d0-4a23-b718-0b59702992e1 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:55.886958 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:55.981234 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9eb9af-e0d0-4a23-b718-0b59702992e1 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:56.016006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5edd9a-5085-435a-91ba-18407230bd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:56.016654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5edd9a-5085-435a-91ba-18407230bd1b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:56.016939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5edd9a-5085-435a-91ba-18407230bd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:56.017241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5edd9a-5085-435a-91ba-18407230bd1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:56.087772 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5edd9a-5085-435a-91ba-18407230bd1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:56.089295 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5edd9a-5085-435a-91ba-18407230bd1b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:56.099013 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:56.190678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 549/1115] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:03:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:03:56.197010 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc60ab1-9dc7-4b1f-a669-a396b56214df None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:56.212018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:56.303478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc60ab1-9dc7-4b1f-a669-a396b56214df None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:56.340382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115bc3e0-0e0e-43d5-bf4a-a112a3e014dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:56.341400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bc3e0-0e0e-43d5-bf4a-a112a3e014dd None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:56.341690 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bc3e0-0e0e-43d5-bf4a-a112a3e014dd None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:56.341973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bc3e0-0e0e-43d5-bf4a-a112a3e014dd None admin] PATH_INFO: `/v3/projects/e5db78f6a088405d997cb70550dc091e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:56.343183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115bc3e0-0e0e-43d5-bf4a-a112a3e014dd None admin] RBAC: Authorizing `identity:get_project(project_id=e5db78f6a088405d997cb70550dc091e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:56.344637 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115bc3e0-0e0e-43d5-bf4a-a112a3e014dd None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:56.346381 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 567/1116] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:03:56 2022] GET /identity/v3/projects/e5db78f6a088405d997cb70550dc091e => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:56.352786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5edd9a-5085-435a-91ba-18407230bd1b None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:56.458610 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5edd9a-5085-435a-91ba-18407230bd1b None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:56.468815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:56.494776 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09f5d79-9252-448d-a3ae-ff4d8e32b941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:56.495519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f5d79-9252-448d-a3ae-ff4d8e32b941 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:56.495803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f5d79-9252-448d-a3ae-ff4d8e32b941 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:56.496081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09f5d79-9252-448d-a3ae-ff4d8e32b941 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:56.496920 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f5d79-9252-448d-a3ae-ff4d8e32b941 None admin] RBAC: Adding query filter params (role.id=d241ebe8f5654f46a5a92be6f6b0cc36, scope.project.id=e5db78f6a088405d997cb70550dc091e, user.id=6241e3a9458f4d629c335692f05fb181) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:03:56.497328 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f5d79-9252-448d-a3ae-ff4d8e32b941 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:56.498763 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09f5d79-9252-448d-a3ae-ff4d8e32b941 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:56.504847 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 550/1117] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:03:56 2022] GET /identity/v3/role_assignments?user.id=6241e3a9458f4d629c335692f05fb181&scope.project.id=e5db78f6a088405d997cb70550dc091e&role.id=d241ebe8f5654f46a5a92be6f6b0cc36 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:58.660830 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00847ac1-3284-416d-89ff-2dab76b8cddc None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:58.661411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00847ac1-3284-416d-89ff-2dab76b8cddc None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:58.661411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00847ac1-3284-416d-89ff-2dab76b8cddc None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:58.663609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 568/1118] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:03:58.669740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f20459-2c83-40a6-b83c-5fbefb62b86f None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:58.670169 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f20459-2c83-40a6-b83c-5fbefb62b86f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:58.670296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f20459-2c83-40a6-b83c-5fbefb62b86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:58.687356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-84f20459-2c83-40a6-b83c-5fbefb62b86f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:03:58.688556 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-84f20459-2c83-40a6-b83c-5fbefb62b86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:03:58.761503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 551/1119] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:03:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:03:58.784766 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00847ac1-3284-416d-89ff-2dab76b8cddc None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:58.868541 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:58.889047 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00847ac1-3284-416d-89ff-2dab76b8cddc None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:58.925700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb6bce-b09b-494d-b9e7-f1353e9b37e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:58.926551 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb6bce-b09b-494d-b9e7-f1353e9b37e2 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:58.926840 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb6bce-b09b-494d-b9e7-f1353e9b37e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:58.927116 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb6bce-b09b-494d-b9e7-f1353e9b37e2 None admin] PATH_INFO: `/v3/roles/d241ebe8f5654f46a5a92be6f6b0cc36` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:58.928601 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb6bce-b09b-494d-b9e7-f1353e9b37e2 None admin] RBAC: Authorizing `identity:get_role(role_id=d241ebe8f5654f46a5a92be6f6b0cc36)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:58.930014 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddb6bce-b09b-494d-b9e7-f1353e9b37e2 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:58.931399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 569/1120] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:03:58 2022] GET /identity/v3/roles/d241ebe8f5654f46a5a92be6f6b0cc36 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:58.937744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f20459-2c83-40a6-b83c-5fbefb62b86f None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:59.045563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f20459-2c83-40a6-b83c-5fbefb62b86f None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:59.047101 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:03:59.082142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb1e25-e504-4aec-b6e9-dfd0a855dac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:59.083013 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb1e25-e504-4aec-b6e9-dfd0a855dac2 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:59.083292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb1e25-e504-4aec-b6e9-dfd0a855dac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:59.083564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb1e25-e504-4aec-b6e9-dfd0a855dac2 None admin] PATH_INFO: `/v3/users/6241e3a9458f4d629c335692f05fb181` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:59.084759 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb1e25-e504-4aec-b6e9-dfd0a855dac2 None admin] RBAC: Authorizing `identity:get_user(user_id=6241e3a9458f4d629c335692f05fb181)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:59.086212 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb1e25-e504-4aec-b6e9-dfd0a855dac2 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:59.087827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 552/1121] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:03:58 2022] GET /identity/v3/users/6241e3a9458f4d629c335692f05fb181 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:59.093856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb6bce-b09b-494d-b9e7-f1353e9b37e2 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:59.195510 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb6bce-b09b-494d-b9e7-f1353e9b37e2 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:59.230480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c2b54-ca38-4ac2-9669-aa18c9df35c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:59.231345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c2b54-ca38-4ac2-9669-aa18c9df35c9 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:59.231630 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c2b54-ca38-4ac2-9669-aa18c9df35c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:59.231904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c2b54-ca38-4ac2-9669-aa18c9df35c9 None admin] PATH_INFO: `/v3/projects/e5db78f6a088405d997cb70550dc091e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:59.233103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c2b54-ca38-4ac2-9669-aa18c9df35c9 None admin] RBAC: Authorizing `identity:get_project(project_id=e5db78f6a088405d997cb70550dc091e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:59.234536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c2b54-ca38-4ac2-9669-aa18c9df35c9 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:59.236236 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 570/1122] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:03:59 2022] GET /identity/v3/projects/e5db78f6a088405d997cb70550dc091e => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:03:59.242223 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb1e25-e504-4aec-b6e9-dfd0a855dac2 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:03:59.349762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb1e25-e504-4aec-b6e9-dfd0a855dac2 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:03:59.384473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590994b-a7bd-4a4a-a426-c3ee3e159534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:03:59.385586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590994b-a7bd-4a4a-a426-c3ee3e159534 None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:03:59.385953 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590994b-a7bd-4a4a-a426-c3ee3e159534 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:03:59.386333 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590994b-a7bd-4a4a-a426-c3ee3e159534 None admin] PATH_INFO: `/v3/projects/e5db78f6a088405d997cb70550dc091e/users/6241e3a9458f4d629c335692f05fb181/roles/d241ebe8f5654f46a5a92be6f6b0cc36` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:03:59.388605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590994b-a7bd-4a4a-a426-c3ee3e159534 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5db78f6a088405d997cb70550dc091e, user_id=6241e3a9458f4d629c335692f05fb181, role_id=d241ebe8f5654f46a5a92be6f6b0cc36)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:03:59.390077 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590994b-a7bd-4a4a-a426-c3ee3e159534 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:03:59.469483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 553/1123] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:03:59 2022] PUT /identity/v3/projects/e5db78f6a088405d997cb70550dc091e/users/6241e3a9458f4d629c335692f05fb181/roles/d241ebe8f5654f46a5a92be6f6b0cc36 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:04:01.606056 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0e1cb-11d3-460c-a2e4-5517f2b7d3cb None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:01.606580 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0e1cb-11d3-460c-a2e4-5517f2b7d3cb None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:01.606580 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0e1cb-11d3-460c-a2e4-5517f2b7d3cb None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:01.607968 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 571/1124] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:04:01.613329 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359275dc-38a8-4478-9890-5b6981098130 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:01.613714 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359275dc-38a8-4478-9890-5b6981098130 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:01.613882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359275dc-38a8-4478-9890-5b6981098130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:01.631055 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-359275dc-38a8-4478-9890-5b6981098130 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:04:01.632234 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-359275dc-38a8-4478-9890-5b6981098130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:04:01.657812 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:01.703564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 554/1125] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:04:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:04:01.727072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0e1cb-11d3-460c-a2e4-5517f2b7d3cb None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:01.750575 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:01.835207 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0e1cb-11d3-460c-a2e4-5517f2b7d3cb None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:01.869206 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d3da5-6251-4072-91b2-cd7e509d79b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:01.870126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d3da5-6251-4072-91b2-cd7e509d79b0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:01.870430 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d3da5-6251-4072-91b2-cd7e509d79b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:01.870709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d3da5-6251-4072-91b2-cd7e509d79b0 None admin] PATH_INFO: `/v3/roles/d241ebe8f5654f46a5a92be6f6b0cc36` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:01.872182 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d3da5-6251-4072-91b2-cd7e509d79b0 None admin] RBAC: Authorizing `identity:get_role(role_id=d241ebe8f5654f46a5a92be6f6b0cc36)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:01.873795 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d3da5-6251-4072-91b2-cd7e509d79b0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:01.875168 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 572/1126] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:04:01 2022] GET /identity/v3/roles/d241ebe8f5654f46a5a92be6f6b0cc36 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:01.881020 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359275dc-38a8-4478-9890-5b6981098130 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:01.979288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359275dc-38a8-4478-9890-5b6981098130 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:02.018468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a84615-0d9d-4d23-9162-f1547d3af18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:02.019338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a84615-0d9d-4d23-9162-f1547d3af18d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:02.019620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a84615-0d9d-4d23-9162-f1547d3af18d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:02.019892 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a84615-0d9d-4d23-9162-f1547d3af18d None admin] PATH_INFO: `/v3/users/6241e3a9458f4d629c335692f05fb181` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:02.021134 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a84615-0d9d-4d23-9162-f1547d3af18d None admin] RBAC: Authorizing `identity:get_user(user_id=6241e3a9458f4d629c335692f05fb181)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:02.022582 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a84615-0d9d-4d23-9162-f1547d3af18d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:02.024239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 555/1127] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:04:01 2022] GET /identity/v3/users/6241e3a9458f4d629c335692f05fb181 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:02.031329 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d3da5-6251-4072-91b2-cd7e509d79b0 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:02.118695 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:02.134179 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d3da5-6251-4072-91b2-cd7e509d79b0 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:02.169924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802f7f7-78fb-4625-8b4a-a7c15079ca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:02.170571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802f7f7-78fb-4625-8b4a-a7c15079ca81 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:02.170852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802f7f7-78fb-4625-8b4a-a7c15079ca81 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:02.171131 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802f7f7-78fb-4625-8b4a-a7c15079ca81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:02.237803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802f7f7-78fb-4625-8b4a-a7c15079ca81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:02.239290 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802f7f7-78fb-4625-8b4a-a7c15079ca81 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:02.321394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:02.339100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 573/1128] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:04:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:04:02.345390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a84615-0d9d-4d23-9162-f1547d3af18d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:02.374365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:02.455486 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a84615-0d9d-4d23-9162-f1547d3af18d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:02.490351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb7409-3841-4813-aa46-28b16bb8da9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:02.491202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb7409-3841-4813-aa46-28b16bb8da9b None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:02.491482 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb7409-3841-4813-aa46-28b16bb8da9b None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:02.491760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb7409-3841-4813-aa46-28b16bb8da9b None admin] PATH_INFO: `/v3/projects/e5db78f6a088405d997cb70550dc091e` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:02.492925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb7409-3841-4813-aa46-28b16bb8da9b None admin] RBAC: Authorizing `identity:get_project(project_id=e5db78f6a088405d997cb70550dc091e)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:02.494378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb7409-3841-4813-aa46-28b16bb8da9b None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:02.496228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 556/1129] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:04:02 2022] GET /identity/v3/projects/e5db78f6a088405d997cb70550dc091e => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:02.502513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802f7f7-78fb-4625-8b4a-a7c15079ca81 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:02.613342 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802f7f7-78fb-4625-8b4a-a7c15079ca81 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:02.647530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b8366-6572-4b14-8b1a-f5f893637566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:02.648116 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b8366-6572-4b14-8b1a-f5f893637566 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:02.648399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b8366-6572-4b14-8b1a-f5f893637566 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:02.648675 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b8366-6572-4b14-8b1a-f5f893637566 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:02.649534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b8366-6572-4b14-8b1a-f5f893637566 None admin] RBAC: Adding query filter params (role.id=d241ebe8f5654f46a5a92be6f6b0cc36, scope.project.id=e5db78f6a088405d997cb70550dc091e, user.id=6241e3a9458f4d629c335692f05fb181) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:04:02.649850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b8366-6572-4b14-8b1a-f5f893637566 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:02.654151 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b8366-6572-4b14-8b1a-f5f893637566 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:02.660452 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 574/1130] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:04:02 2022] GET /identity/v3/role_assignments?user.id=6241e3a9458f4d629c335692f05fb181&scope.project.id=e5db78f6a088405d997cb70550dc091e&role.id=d241ebe8f5654f46a5a92be6f6b0cc36 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:04.835131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc446b-6966-48fc-a151-36e52acf87dc None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:04.835648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc446b-6966-48fc-a151-36e52acf87dc None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:04.835648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc446b-6966-48fc-a151-36e52acf87dc None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:04.836981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 557/1131] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:04:04.842600 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2f2d2-fac5-4c6e-89ed-04311e973c61 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:04.842994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2f2d2-fac5-4c6e-89ed-04311e973c61 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:04.843157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2f2d2-fac5-4c6e-89ed-04311e973c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:04.860085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-5ad2f2d2-fac5-4c6e-89ed-04311e973c61 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:04:04.861294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5ad2f2d2-fac5-4c6e-89ed-04311e973c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:04:04.904134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:04.926671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 575/1132] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:04:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:04:04.950392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc446b-6966-48fc-a151-36e52acf87dc None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:05.042712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:05.049122 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc446b-6966-48fc-a151-36e52acf87dc None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:05.082427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b16da-3802-4d0f-8bdc-50f1afa2cfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:05.083217 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b16da-3802-4d0f-8bdc-50f1afa2cfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:05.083499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b16da-3802-4d0f-8bdc-50f1afa2cfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:05.083780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b16da-3802-4d0f-8bdc-50f1afa2cfc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:05.085007 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b16da-3802-4d0f-8bdc-50f1afa2cfc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:05.086442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b16da-3802-4d0f-8bdc-50f1afa2cfc9 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:05.087999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 558/1133] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:04:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:05.094425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2f2d2-fac5-4c6e-89ed-04311e973c61 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:05.190464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2f2d2-fac5-4c6e-89ed-04311e973c61 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:05.224146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785449fc-8932-4c30-a4fa-b59b6d08967b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:05.224812 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785449fc-8932-4c30-a4fa-b59b6d08967b None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:05.225130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785449fc-8932-4c30-a4fa-b59b6d08967b None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:05.225406 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785449fc-8932-4c30-a4fa-b59b6d08967b None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:05.226122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785449fc-8932-4c30-a4fa-b59b6d08967b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:05.227555 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785449fc-8932-4c30-a4fa-b59b6d08967b None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:05.310165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 576/1134] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jun 15 10:04:05 2022] POST /identity/v3/projects => generated 311 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:04:07.489173 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58888399-956e-4e1e-a088-fabd47bd57a3 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:07.489728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58888399-956e-4e1e-a088-fabd47bd57a3 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:07.489728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58888399-956e-4e1e-a088-fabd47bd57a3 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:07.491080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 559/1135] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:04:07.497075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef44e2-aa52-4a6e-890c-b6e7624bb6c6 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:07.497451 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef44e2-aa52-4a6e-890c-b6e7624bb6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:07.497647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef44e2-aa52-4a6e-890c-b6e7624bb6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:07.515043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-f8ef44e2-aa52-4a6e-890c-b6e7624bb6c6 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:04:07.516264 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f8ef44e2-aa52-4a6e-890c-b6e7624bb6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:04:07.543195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:07.592724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 577/1136] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:04:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:04:07.618037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58888399-956e-4e1e-a088-fabd47bd57a3 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:07.726534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58888399-956e-4e1e-a088-fabd47bd57a3 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:07.761732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f1d83-3715-4b8c-a239-14d7d6767ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:07.762429 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f1d83-3715-4b8c-a239-14d7d6767ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:07.762716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f1d83-3715-4b8c-a239-14d7d6767ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:07.762996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f1d83-3715-4b8c-a239-14d7d6767ae5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:07.764191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f1d83-3715-4b8c-a239-14d7d6767ae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:07.765661 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f1d83-3715-4b8c-a239-14d7d6767ae5 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:07.767283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 560/1137] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:04:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:07.773090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef44e2-aa52-4a6e-890c-b6e7624bb6c6 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:07.871021 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef44e2-aa52-4a6e-890c-b6e7624bb6c6 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:07.913885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e780803-227e-4731-a9f9-39a4af07235b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:07.914556 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e780803-227e-4731-a9f9-39a4af07235b None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:07.914836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e780803-227e-4731-a9f9-39a4af07235b None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:07.915107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e780803-227e-4731-a9f9-39a4af07235b None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:07.915808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e780803-227e-4731-a9f9-39a4af07235b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:07.917295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e780803-227e-4731-a9f9-39a4af07235b None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:08.017439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 578/1138] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:04:07 2022] POST /identity/v3/users => generated 286 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:04:10.231748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8ff8d-cc72-48a1-8252-a1dd59fb1415 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:10.232316 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8ff8d-cc72-48a1-8252-a1dd59fb1415 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:10.232316 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8ff8d-cc72-48a1-8252-a1dd59fb1415 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:10.233858 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 561/1139] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:04:10.239162 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3abed-716a-44fd-bced-ee3f1fc7e75d None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:10.239561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3abed-716a-44fd-bced-ee3f1fc7e75d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:10.239731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3abed-716a-44fd-bced-ee3f1fc7e75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:10.256533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-9ca3abed-716a-44fd-bced-ee3f1fc7e75d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:04:10.257746 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9ca3abed-716a-44fd-bced-ee3f1fc7e75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:04:10.269121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:10.328405 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 579/1140] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:04:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:04:10.352167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8ff8d-cc72-48a1-8252-a1dd59fb1415 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:10.382824 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:10.460585 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8ff8d-cc72-48a1-8252-a1dd59fb1415 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:10.497356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785aaffe-633a-473a-9121-1cea60109154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:10.498069 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aaffe-633a-473a-9121-1cea60109154 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:10.498348 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aaffe-633a-473a-9121-1cea60109154 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:10.498620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aaffe-633a-473a-9121-1cea60109154 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:10.508640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785aaffe-633a-473a-9121-1cea60109154 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-785aaffe-633a-473a-9121-1cea60109154 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:04:10.517243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:04:10.518324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:04:10.519191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:04:10.519191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:04:10.519191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:04:10.519191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:04:10.519191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:04:10.519191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:04:10.519191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:04:10.519191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:04:10.519191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:04:10.519191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:04:10.519191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:04:10.519191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:04:10.519191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 562/1141] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:04:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:04:10.524657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3abed-716a-44fd-bced-ee3f1fc7e75d None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:10.623909 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3abed-716a-44fd-bced-ee3f1fc7e75d None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:10.657975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a444dd-f7b3-4a66-8186-d777d049fc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:10.658648 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a444dd-f7b3-4a66-8186-d777d049fc5c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:10.658937 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a444dd-f7b3-4a66-8186-d777d049fc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:10.659219 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a444dd-f7b3-4a66-8186-d777d049fc5c None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:10.659986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a444dd-f7b3-4a66-8186-d777d049fc5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:04:10.660297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a444dd-f7b3-4a66-8186-d777d049fc5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:10.661705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a444dd-f7b3-4a66-8186-d777d049fc5c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:10.670577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 580/1142] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:04:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:10.676818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785aaffe-633a-473a-9121-1cea60109154 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:10.768029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:10.777481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785aaffe-633a-473a-9121-1cea60109154 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:10.812184 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d32200-e0a8-46ab-bac0-3752c4d10de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:10.813098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d32200-e0a8-46ab-bac0-3752c4d10de8 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:10.813382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d32200-e0a8-46ab-bac0-3752c4d10de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:10.813655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d32200-e0a8-46ab-bac0-3752c4d10de8 None admin] PATH_INFO: `/v3/users/6a5f685dd2ba4d9f9dfa8afaad7753d8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:10.827142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d32200-e0a8-46ab-bac0-3752c4d10de8 None admin] RBAC: Authorizing `identity:get_user(user_id=6a5f685dd2ba4d9f9dfa8afaad7753d8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:10.828571 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d32200-e0a8-46ab-bac0-3752c4d10de8 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:10.830271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 563/1143] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:04:10 2022] GET /identity/v3/users/6a5f685dd2ba4d9f9dfa8afaad7753d8 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:10.837179 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a444dd-f7b3-4a66-8186-d777d049fc5c None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:10.848976 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:10.936831 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a444dd-f7b3-4a66-8186-d777d049fc5c None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:10.972233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2928b-3535-4b4a-9dd7-ee9d8e5cffd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:10.972888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2928b-3535-4b4a-9dd7-ee9d8e5cffd2 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:10.973200 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2928b-3535-4b4a-9dd7-ee9d8e5cffd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:10.973479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2928b-3535-4b4a-9dd7-ee9d8e5cffd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:11.042290 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2928b-3535-4b4a-9dd7-ee9d8e5cffd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:11.043686 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2928b-3535-4b4a-9dd7-ee9d8e5cffd2 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:11.053747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:11.141423 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 581/1144] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:04:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:04:11.147361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d32200-e0a8-46ab-bac0-3752c4d10de8 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:11.247147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d32200-e0a8-46ab-bac0-3752c4d10de8 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:11.281726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc89a1-1b57-485b-9a0c-e1dc1b0ede2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:11.282578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc89a1-1b57-485b-9a0c-e1dc1b0ede2d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:11.282855 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc89a1-1b57-485b-9a0c-e1dc1b0ede2d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:11.283124 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc89a1-1b57-485b-9a0c-e1dc1b0ede2d None admin] PATH_INFO: `/v3/projects/fe341c1aa58e492980774a6c55dfcbee` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:11.284284 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc89a1-1b57-485b-9a0c-e1dc1b0ede2d None admin] RBAC: Authorizing `identity:get_project(project_id=fe341c1aa58e492980774a6c55dfcbee)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:11.285726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc89a1-1b57-485b-9a0c-e1dc1b0ede2d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:11.287413 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 564/1145] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:04:11 2022] GET /identity/v3/projects/fe341c1aa58e492980774a6c55dfcbee => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:11.294280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2928b-3535-4b4a-9dd7-ee9d8e5cffd2 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:11.395291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2928b-3535-4b4a-9dd7-ee9d8e5cffd2 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:11.405072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:11.430174 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87428571-d882-4a17-8df8-d331170f134d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:11.430828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87428571-d882-4a17-8df8-d331170f134d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:11.431109 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87428571-d882-4a17-8df8-d331170f134d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:11.431388 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87428571-d882-4a17-8df8-d331170f134d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:11.432494 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87428571-d882-4a17-8df8-d331170f134d None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=fe341c1aa58e492980774a6c55dfcbee, user.id=6a5f685dd2ba4d9f9dfa8afaad7753d8) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:04:11.432807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87428571-d882-4a17-8df8-d331170f134d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:11.434271 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87428571-d882-4a17-8df8-d331170f134d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:11.440433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 582/1146] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:04:11 2022] GET /identity/v3/role_assignments?user.id=6a5f685dd2ba4d9f9dfa8afaad7753d8&scope.project.id=fe341c1aa58e492980774a6c55dfcbee&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:13.589114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb1d43-72c1-4d45-8355-925fce0b40fc None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:13.589625 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb1d43-72c1-4d45-8355-925fce0b40fc None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:13.589742 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb1d43-72c1-4d45-8355-925fce0b40fc None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:13.591029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 565/1147] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:04:13.596371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd2013-48f0-4ece-97d9-7b109044037c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:13.596793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd2013-48f0-4ece-97d9-7b109044037c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:13.596943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd2013-48f0-4ece-97d9-7b109044037c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:13.614550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-08fd2013-48f0-4ece-97d9-7b109044037c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:04:13.615772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-08fd2013-48f0-4ece-97d9-7b109044037c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:04:13.683271 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 583/1148] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:04:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:04:13.706709 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb1d43-72c1-4d45-8355-925fce0b40fc None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:13.725624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:13.807945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb1d43-72c1-4d45-8355-925fce0b40fc None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:13.841396 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa0be3-0e92-461c-8e2d-172665d79440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:13.842175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa0be3-0e92-461c-8e2d-172665d79440 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:13.842459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa0be3-0e92-461c-8e2d-172665d79440 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:13.842733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa0be3-0e92-461c-8e2d-172665d79440 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:13.852780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa0be3-0e92-461c-8e2d-172665d79440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-d1fa0be3-0e92-461c-8e2d-172665d79440 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:04:13.861366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:04:13.862455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:04:13.863491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:04:13.863491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:04:13.863491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:04:13.863491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:04:13.863491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:04:13.863491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:04:13.863491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:04:13.863491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:04:13.863491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:04:13.863491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:04:13.863491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:04:13.863491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:04:13.863491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 566/1149] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:04:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:04:13.868792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd2013-48f0-4ece-97d9-7b109044037c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:13.976231 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd2013-48f0-4ece-97d9-7b109044037c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:13.977767 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:14.010404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e4b6e-8141-4219-9415-0572ca8cb648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:14.011370 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e4b6e-8141-4219-9415-0572ca8cb648 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:14.011733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e4b6e-8141-4219-9415-0572ca8cb648 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:14.012382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e4b6e-8141-4219-9415-0572ca8cb648 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:14.013177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e4b6e-8141-4219-9415-0572ca8cb648 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:04:14.013497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e4b6e-8141-4219-9415-0572ca8cb648 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:14.014886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e4b6e-8141-4219-9415-0572ca8cb648 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:14.023414 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 584/1150] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:04:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:14.029952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa0be3-0e92-461c-8e2d-172665d79440 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:14.108414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:14.129428 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa0be3-0e92-461c-8e2d-172665d79440 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:14.162479 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d97ec-e618-4436-bc05-10e65454466e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:14.163355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d97ec-e618-4436-bc05-10e65454466e None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:14.163631 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d97ec-e618-4436-bc05-10e65454466e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:14.163899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d97ec-e618-4436-bc05-10e65454466e None admin] PATH_INFO: `/v3/users/6a5f685dd2ba4d9f9dfa8afaad7753d8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:14.165102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d97ec-e618-4436-bc05-10e65454466e None admin] RBAC: Authorizing `identity:get_user(user_id=6a5f685dd2ba4d9f9dfa8afaad7753d8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:14.167288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d97ec-e618-4436-bc05-10e65454466e None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:14.168911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 567/1151] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:04:14 2022] GET /identity/v3/users/6a5f685dd2ba4d9f9dfa8afaad7753d8 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:14.175020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e4b6e-8141-4219-9415-0572ca8cb648 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:14.274708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e4b6e-8141-4219-9415-0572ca8cb648 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:14.309604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ce6f2-0d71-4921-8b14-e451420be409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:14.310475 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ce6f2-0d71-4921-8b14-e451420be409 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:14.310917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ce6f2-0d71-4921-8b14-e451420be409 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:14.311206 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ce6f2-0d71-4921-8b14-e451420be409 None admin] PATH_INFO: `/v3/projects/fe341c1aa58e492980774a6c55dfcbee` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:14.312389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ce6f2-0d71-4921-8b14-e451420be409 None admin] RBAC: Authorizing `identity:get_project(project_id=fe341c1aa58e492980774a6c55dfcbee)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:14.313854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ce6f2-0d71-4921-8b14-e451420be409 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:14.315553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 585/1152] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:04:14 2022] GET /identity/v3/projects/fe341c1aa58e492980774a6c55dfcbee => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:14.321532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d97ec-e618-4436-bc05-10e65454466e None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:14.426533 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d97ec-e618-4436-bc05-10e65454466e None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:14.458768 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359cbab-0a4c-4c01-98c3-fe9cfd4e4011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:14.459649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359cbab-0a4c-4c01-98c3-fe9cfd4e4011 None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:14.459922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359cbab-0a4c-4c01-98c3-fe9cfd4e4011 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:14.460188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359cbab-0a4c-4c01-98c3-fe9cfd4e4011 None admin] PATH_INFO: `/v3/projects/fe341c1aa58e492980774a6c55dfcbee/users/6a5f685dd2ba4d9f9dfa8afaad7753d8/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:14.462119 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359cbab-0a4c-4c01-98c3-fe9cfd4e4011 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe341c1aa58e492980774a6c55dfcbee, user_id=6a5f685dd2ba4d9f9dfa8afaad7753d8, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:14.464064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359cbab-0a4c-4c01-98c3-fe9cfd4e4011 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:14.551825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 568/1153] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:04:14 2022] PUT /identity/v3/projects/fe341c1aa58e492980774a6c55dfcbee/users/6a5f685dd2ba4d9f9dfa8afaad7753d8/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:04:16.707785 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123dda5-d135-4409-8d7b-238077ce41fb None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:16.708306 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123dda5-d135-4409-8d7b-238077ce41fb None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:16.708412 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123dda5-d135-4409-8d7b-238077ce41fb None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:16.709743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 586/1154] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:04:16.715587 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86bf30-f903-4746-9b13-357d4971cddf None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:16.715985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86bf30-f903-4746-9b13-357d4971cddf None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:16.716149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86bf30-f903-4746-9b13-357d4971cddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:16.727848 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:16.734188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-0f86bf30-f903-4746-9b13-357d4971cddf None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:04:16.735403 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-0f86bf30-f903-4746-9b13-357d4971cddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:04:16.804477 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 569/1155] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:04:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:04:16.828162 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123dda5-d135-4409-8d7b-238077ce41fb None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:16.860234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:16.932363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123dda5-d135-4409-8d7b-238077ce41fb None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:16.965803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e1729-448a-471c-aaa8-34132245b7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:16.966595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e1729-448a-471c-aaa8-34132245b7ca None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:16.966879 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e1729-448a-471c-aaa8-34132245b7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:16.967151 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e1729-448a-471c-aaa8-34132245b7ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:16.977630 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e1729-448a-471c-aaa8-34132245b7ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-ed8e1729-448a-471c-aaa8-34132245b7ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:04:16.986082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:04:16.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:04:16.988087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:04:16.988087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:04:16.988087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:04:16.988087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:04:16.988087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:04:16.988087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:04:16.988087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:04:16.988087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:04:16.988087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:04:16.988087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:04:16.988087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:04:16.988087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:04:16.988087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 587/1156] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:04:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:04:16.995358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86bf30-f903-4746-9b13-357d4971cddf None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:17.093987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86bf30-f903-4746-9b13-357d4971cddf None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:17.130506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e7724-6526-4c4a-bdf5-3864057edf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:17.131167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e7724-6526-4c4a-bdf5-3864057edf49 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:17.131447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e7724-6526-4c4a-bdf5-3864057edf49 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:17.131717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e7724-6526-4c4a-bdf5-3864057edf49 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:17.132466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e7724-6526-4c4a-bdf5-3864057edf49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:04:17.132772 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e7724-6526-4c4a-bdf5-3864057edf49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:17.134156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e7724-6526-4c4a-bdf5-3864057edf49 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:17.136492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:17.142933 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 570/1157] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:04:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:17.149172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e1729-448a-471c-aaa8-34132245b7ca None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:17.239075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:17.248805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e1729-448a-471c-aaa8-34132245b7ca None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:17.283021 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899998b5-f8ef-4202-9b8c-6b5711c470fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:17.283890 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899998b5-f8ef-4202-9b8c-6b5711c470fc None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:17.284168 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899998b5-f8ef-4202-9b8c-6b5711c470fc None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:17.284441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899998b5-f8ef-4202-9b8c-6b5711c470fc None admin] PATH_INFO: `/v3/users/6a5f685dd2ba4d9f9dfa8afaad7753d8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:17.285687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899998b5-f8ef-4202-9b8c-6b5711c470fc None admin] RBAC: Authorizing `identity:get_user(user_id=6a5f685dd2ba4d9f9dfa8afaad7753d8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:17.287226 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899998b5-f8ef-4202-9b8c-6b5711c470fc None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:17.288849 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 588/1158] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:04:17 2022] GET /identity/v3/users/6a5f685dd2ba4d9f9dfa8afaad7753d8 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:17.295588 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e7724-6526-4c4a-bdf5-3864057edf49 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:17.398559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e7724-6526-4c4a-bdf5-3864057edf49 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:17.434203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb81d3-614c-4569-b84b-bdf52e97a857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:17.434841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb81d3-614c-4569-b84b-bdf52e97a857 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:17.435209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb81d3-614c-4569-b84b-bdf52e97a857 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:17.435393 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb81d3-614c-4569-b84b-bdf52e97a857 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:17.499132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:17.502553 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb81d3-614c-4569-b84b-bdf52e97a857 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:17.503988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb81d3-614c-4569-b84b-bdf52e97a857 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:17.601971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 571/1159] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:04:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:04:17.608760 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899998b5-f8ef-4202-9b8c-6b5711c470fc None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:17.708750 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899998b5-f8ef-4202-9b8c-6b5711c470fc None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:17.743179 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d491990-2268-4a1a-b71f-93c91de24f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:17.744262 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d491990-2268-4a1a-b71f-93c91de24f19 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:17.744546 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d491990-2268-4a1a-b71f-93c91de24f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:17.744823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d491990-2268-4a1a-b71f-93c91de24f19 None admin] PATH_INFO: `/v3/projects/fe341c1aa58e492980774a6c55dfcbee` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:17.746036 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d491990-2268-4a1a-b71f-93c91de24f19 None admin] RBAC: Authorizing `identity:get_project(project_id=fe341c1aa58e492980774a6c55dfcbee)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:17.747461 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d491990-2268-4a1a-b71f-93c91de24f19 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:17.749187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 589/1160] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:04:17 2022] GET /identity/v3/projects/fe341c1aa58e492980774a6c55dfcbee => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:17.755454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb81d3-614c-4569-b84b-bdf52e97a857 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:17.850782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:17.854957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb81d3-614c-4569-b84b-bdf52e97a857 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:17.892147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519b5fa-5f7a-48dd-8414-209c030ed97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:17.892796 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519b5fa-5f7a-48dd-8414-209c030ed97c None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:17.893112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519b5fa-5f7a-48dd-8414-209c030ed97c None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:17.893387 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519b5fa-5f7a-48dd-8414-209c030ed97c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:17.894219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519b5fa-5f7a-48dd-8414-209c030ed97c None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=fe341c1aa58e492980774a6c55dfcbee, user.id=6a5f685dd2ba4d9f9dfa8afaad7753d8) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:04:17.894529 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519b5fa-5f7a-48dd-8414-209c030ed97c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:17.895942 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519b5fa-5f7a-48dd-8414-209c030ed97c None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:17.902629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 572/1161] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:04:17 2022] GET /identity/v3/role_assignments?user.id=6a5f685dd2ba4d9f9dfa8afaad7753d8&scope.project.id=fe341c1aa58e492980774a6c55dfcbee&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:20.054603 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8aa8-cef2-4bd0-b695-629217a44948 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:20.055188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8aa8-cef2-4bd0-b695-629217a44948 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:20.055188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba8aa8-cef2-4bd0-b695-629217a44948 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:20.056744 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 590/1162] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:04:20.062669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d531a-1aa9-4d03-bbf5-74f269d2c7e7 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:20.063088 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d531a-1aa9-4d03-bbf5-74f269d2c7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:20.063222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d531a-1aa9-4d03-bbf5-74f269d2c7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:20.079551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-063d531a-1aa9-4d03-bbf5-74f269d2c7e7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:04:20.080727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-063d531a-1aa9-4d03-bbf5-74f269d2c7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:04:20.145727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 573/1163] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:04:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:04:20.169055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba8aa8-cef2-4bd0-b695-629217a44948 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:20.197509 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:20.269831 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba8aa8-cef2-4bd0-b695-629217a44948 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:20.303503 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ab397-b0e7-4e1c-9007-eb8556e9827f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:20.304149 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ab397-b0e7-4e1c-9007-eb8556e9827f None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:20.304435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ab397-b0e7-4e1c-9007-eb8556e9827f None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:20.304766 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ab397-b0e7-4e1c-9007-eb8556e9827f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:20.371722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ab397-b0e7-4e1c-9007-eb8556e9827f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:20.373227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ab397-b0e7-4e1c-9007-eb8556e9827f None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:20.400730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:20.470259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 591/1164] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:04:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:04:20.476251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d531a-1aa9-4d03-bbf5-74f269d2c7e7 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:20.565548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:20.573936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d531a-1aa9-4d03-bbf5-74f269d2c7e7 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:20.607136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1442c7a-92af-4819-8817-c92f2514700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:20.608161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1442c7a-92af-4819-8817-c92f2514700a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:20.608635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1442c7a-92af-4819-8817-c92f2514700a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:20.609245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1442c7a-92af-4819-8817-c92f2514700a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:20.615771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1442c7a-92af-4819-8817-c92f2514700a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:20.617220 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1442c7a-92af-4819-8817-c92f2514700a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-d1442c7a-92af-4819-8817-c92f2514700a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:04:20.631109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:04:20.632456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:04:20.633592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:04:20.633592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:04:20.633592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:04:20.633592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:04:20.633592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:04:20.633592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:04:20.633592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:04:20.633592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:04:20.633592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:04:20.633592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:04:20.633592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 15 10:04:20.633592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:04:20.633592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 15 10:04:20.633592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:04:20.633592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 574/1165] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jun 15 10:04:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:04:20.639660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ab397-b0e7-4e1c-9007-eb8556e9827f None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:20.736716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ab397-b0e7-4e1c-9007-eb8556e9827f None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:20.762267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:20.769593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e2dec-9103-4750-8673-1e674ca33f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:20.770250 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e2dec-9103-4750-8673-1e674ca33f1a None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:20.770533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e2dec-9103-4750-8673-1e674ca33f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:20.770811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e2dec-9103-4750-8673-1e674ca33f1a None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:20.771529 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e2dec-9103-4750-8673-1e674ca33f1a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:04:20.771835 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e2dec-9103-4750-8673-1e674ca33f1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:20.773239 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e2dec-9103-4750-8673-1e674ca33f1a None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:20.780831 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 592/1166] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Jun 15 10:04:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:22.910217 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a324ac-1fac-41c5-9679-2afd1fd6b8db None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:22.910759 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a324ac-1fac-41c5-9679-2afd1fd6b8db None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:22.910759 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a324ac-1fac-41c5-9679-2afd1fd6b8db None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:22.912097 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 575/1167] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:04:22.918881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074218a-f3b4-4f4f-aac4-6fef0cacde7c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:22.919404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074218a-f3b4-4f4f-aac4-6fef0cacde7c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:22.919516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d074218a-f3b4-4f4f-aac4-6fef0cacde7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:22.937392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-d074218a-f3b4-4f4f-aac4-6fef0cacde7c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:04:22.938596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d074218a-f3b4-4f4f-aac4-6fef0cacde7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:04:23.005368 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 593/1168] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:04:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:04:23.028566 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a324ac-1fac-41c5-9679-2afd1fd6b8db None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:23.133062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a324ac-1fac-41c5-9679-2afd1fd6b8db None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:23.164915 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a00bd-c89a-432e-a918-a28ee425cdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:23.166066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a00bd-c89a-432e-a918-a28ee425cdb4 None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:23.166347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a00bd-c89a-432e-a918-a28ee425cdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:23.166622 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a00bd-c89a-432e-a918-a28ee425cdb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:23.167378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a00bd-c89a-432e-a918-a28ee425cdb4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:23.168734 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a00bd-c89a-432e-a918-a28ee425cdb4 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:23.249306 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 576/1169] 10.222.0.60 () {60 vars in 1138 bytes} [Wed Jun 15 10:04:23 2022] POST /identity/v3/domains => generated 267 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:04:25.406195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ff62a-4382-4e83-99b7-603c0b844179 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:25.406750 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ff62a-4382-4e83-99b7-603c0b844179 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:25.406750 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ff62a-4382-4e83-99b7-603c0b844179 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:25.408107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 594/1170] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:04:25.413193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d0f7a-153b-4635-8128-3012a0686139 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:25.413636 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d0f7a-153b-4635-8128-3012a0686139 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:25.413729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d0f7a-153b-4635-8128-3012a0686139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:25.431596 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-940d0f7a-153b-4635-8128-3012a0686139 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:04:25.432776 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-940d0f7a-153b-4635-8128-3012a0686139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:04:25.439842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:25.499854 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 577/1171] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:04:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:04:25.523522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ff62a-4382-4e83-99b7-603c0b844179 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:25.622377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ff62a-4382-4e83-99b7-603c0b844179 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:25.644761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:25.655947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0de176-90b6-41f9-81f6-9ccaadde3f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:25.656859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0de176-90b6-41f9-81f6-9ccaadde3f3a None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:25.657117 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0de176-90b6-41f9-81f6-9ccaadde3f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:25.657394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0de176-90b6-41f9-81f6-9ccaadde3f3a None admin] PATH_INFO: `/v3/domains/b5cf7ba847a14434ae2df378a4e9db73` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:25.667530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0de176-90b6-41f9-81f6-9ccaadde3f3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=b5cf7ba847a14434ae2df378a4e9db73)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:25.669000 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0de176-90b6-41f9-81f6-9ccaadde3f3a None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:25.670907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 595/1172] 10.222.0.60 () {56 vars in 1183 bytes} [Wed Jun 15 10:04:25 2022] GET /identity/v3/domains/b5cf7ba847a14434ae2df378a4e9db73 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:25.676770 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d0f7a-153b-4635-8128-3012a0686139 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:25.773882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d0f7a-153b-4635-8128-3012a0686139 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:25.807694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e0592-5e97-4a93-8653-6b9ab477dc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:25.808344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e0592-5e97-4a93-8653-6b9ab477dc20 None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:25.808629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e0592-5e97-4a93-8653-6b9ab477dc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:25.808908 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e0592-5e97-4a93-8653-6b9ab477dc20 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:25.809647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e0592-5e97-4a93-8653-6b9ab477dc20 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:25.811075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e0592-5e97-4a93-8653-6b9ab477dc20 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:25.823998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:25.890860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 578/1173] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jun 15 10:04:25 2022] POST /identity/v3/projects => generated 361 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:04:28.049425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057780e-fed3-42b0-859d-c259e4630b17 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:28.049967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057780e-fed3-42b0-859d-c259e4630b17 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:28.049967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057780e-fed3-42b0-859d-c259e4630b17 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:28.051336 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 596/1174] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:04:28.057061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1829aa-fa50-4b6d-b8f9-2d892f2cc632 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:28.057452 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1829aa-fa50-4b6d-b8f9-2d892f2cc632 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:28.057772 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1829aa-fa50-4b6d-b8f9-2d892f2cc632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:28.074775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-ea1829aa-fa50-4b6d-b8f9-2d892f2cc632 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:04:28.075952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ea1829aa-fa50-4b6d-b8f9-2d892f2cc632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:04:28.142382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 579/1175] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:04:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:04:28.169339 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057780e-fed3-42b0-859d-c259e4630b17 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:28.273221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057780e-fed3-42b0-859d-c259e4630b17 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:28.306484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fed4ed-b892-4773-a3c2-8f9e2b3f6c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:28.307361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fed4ed-b892-4773-a3c2-8f9e2b3f6c93 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:28.307646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fed4ed-b892-4773-a3c2-8f9e2b3f6c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:28.307925 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fed4ed-b892-4773-a3c2-8f9e2b3f6c93 None admin] PATH_INFO: `/v3/domains/b5cf7ba847a14434ae2df378a4e9db73` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:28.309148 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fed4ed-b892-4773-a3c2-8f9e2b3f6c93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b5cf7ba847a14434ae2df378a4e9db73)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:28.310582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fed4ed-b892-4773-a3c2-8f9e2b3f6c93 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:28.312272 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 597/1176] 10.222.0.60 () {56 vars in 1183 bytes} [Wed Jun 15 10:04:28 2022] GET /identity/v3/domains/b5cf7ba847a14434ae2df378a4e9db73 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:28.318128 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1829aa-fa50-4b6d-b8f9-2d892f2cc632 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:28.411918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1829aa-fa50-4b6d-b8f9-2d892f2cc632 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:28.436592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:28.447293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4ecd0-d11e-42f9-aa51-c40ee72f5908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:28.449259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4ecd0-d11e-42f9-aa51-c40ee72f5908 None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:28.449259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4ecd0-d11e-42f9-aa51-c40ee72f5908 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:28.449259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4ecd0-d11e-42f9-aa51-c40ee72f5908 None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:28.449512 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4ecd0-d11e-42f9-aa51-c40ee72f5908 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:28.450900 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4ecd0-d11e-42f9-aa51-c40ee72f5908 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:28.542329 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 580/1177] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:04:28 2022] POST /identity/v3/users => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:04:30.706121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebb8f6-95bb-46ec-b0b1-606b953dfb14 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:30.706664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebb8f6-95bb-46ec-b0b1-606b953dfb14 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:30.706664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebb8f6-95bb-46ec-b0b1-606b953dfb14 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:30.708034 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 598/1178] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:04:30.713739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d756c-43af-4b95-8216-1fdd1e36ae06 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:30.714144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d756c-43af-4b95-8216-1fdd1e36ae06 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:30.714294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d756c-43af-4b95-8216-1fdd1e36ae06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:30.731301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-e60d756c-43af-4b95-8216-1fdd1e36ae06 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:04:30.732473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e60d756c-43af-4b95-8216-1fdd1e36ae06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:04:30.797574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 581/1179] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:04:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:04:30.821721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebb8f6-95bb-46ec-b0b1-606b953dfb14 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:30.875459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:30.925181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebb8f6-95bb-46ec-b0b1-606b953dfb14 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:30.958083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173c2b2-3bb2-4bad-9804-271f80f26207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:30.958865 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173c2b2-3bb2-4bad-9804-271f80f26207 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:30.959146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173c2b2-3bb2-4bad-9804-271f80f26207 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:30.959420 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173c2b2-3bb2-4bad-9804-271f80f26207 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:30.969825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173c2b2-3bb2-4bad-9804-271f80f26207 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-f173c2b2-3bb2-4bad-9804-271f80f26207 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:04:30.977449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:04:30.979203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:04:30.980228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:04:30.980228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:04:30.980228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:04:30.980228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:04:30.980228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:04:30.980228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:04:30.980228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:04:30.980228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:04:30.980228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:04:30.980228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:04:30.980228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:04:30.980228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:04:30.980228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 599/1180] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:04:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:04:30.985424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d756c-43af-4b95-8216-1fdd1e36ae06 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:31.080171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d756c-43af-4b95-8216-1fdd1e36ae06 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:31.091204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:31.115621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871a2c4-74b5-4d21-846b-fff345992dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:31.116289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871a2c4-74b5-4d21-846b-fff345992dde None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:31.116572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871a2c4-74b5-4d21-846b-fff345992dde None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:31.116846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871a2c4-74b5-4d21-846b-fff345992dde None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:31.117635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871a2c4-74b5-4d21-846b-fff345992dde None admin] RBAC: Adding query filter params (name=admin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:04:31.117966 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871a2c4-74b5-4d21-846b-fff345992dde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:31.119371 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871a2c4-74b5-4d21-846b-fff345992dde None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:31.127914 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 582/1181] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:04:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:31.134465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173c2b2-3bb2-4bad-9804-271f80f26207 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:31.237842 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173c2b2-3bb2-4bad-9804-271f80f26207 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:31.248724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:31.272330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299f1f5-5ce2-43d5-a6de-6b43d6120dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:31.273238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299f1f5-5ce2-43d5-a6de-6b43d6120dbe None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:31.273524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299f1f5-5ce2-43d5-a6de-6b43d6120dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:31.273803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299f1f5-5ce2-43d5-a6de-6b43d6120dbe None admin] PATH_INFO: `/v3/users/a220c43e5ca041a5a8435460d287c4d9` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:31.287758 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299f1f5-5ce2-43d5-a6de-6b43d6120dbe None admin] RBAC: Authorizing `identity:get_user(user_id=a220c43e5ca041a5a8435460d287c4d9)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:31.289244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299f1f5-5ce2-43d5-a6de-6b43d6120dbe None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:31.290989 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 600/1182] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:04:31 2022] GET /identity/v3/users/a220c43e5ca041a5a8435460d287c4d9 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:31.297922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871a2c4-74b5-4d21-846b-fff345992dde None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:31.399116 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871a2c4-74b5-4d21-846b-fff345992dde None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:31.432223 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0b814-e926-4fb2-b8d3-05898c1d4af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:31.432862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0b814-e926-4fb2-b8d3-05898c1d4af6 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:31.433167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0b814-e926-4fb2-b8d3-05898c1d4af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:31.433443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0b814-e926-4fb2-b8d3-05898c1d4af6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:31.474146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:31.500449 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0b814-e926-4fb2-b8d3-05898c1d4af6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:31.502290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0b814-e926-4fb2-b8d3-05898c1d4af6 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:31.600426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 583/1183] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:04:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:04:31.606579 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299f1f5-5ce2-43d5-a6de-6b43d6120dbe None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:31.702645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299f1f5-5ce2-43d5-a6de-6b43d6120dbe None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:31.735967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05f6e9-8978-4b55-8062-0e0ef8381225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:31.736826 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05f6e9-8978-4b55-8062-0e0ef8381225 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:31.737141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05f6e9-8978-4b55-8062-0e0ef8381225 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:31.737421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05f6e9-8978-4b55-8062-0e0ef8381225 None admin] PATH_INFO: `/v3/projects/ff03ff4de1984e37b2cf45e648908f15` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:31.738601 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05f6e9-8978-4b55-8062-0e0ef8381225 None admin] RBAC: Authorizing `identity:get_project(project_id=ff03ff4de1984e37b2cf45e648908f15)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:31.740034 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05f6e9-8978-4b55-8062-0e0ef8381225 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:31.741753 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 601/1184] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:04:31 2022] GET /identity/v3/projects/ff03ff4de1984e37b2cf45e648908f15 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:31.748390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0b814-e926-4fb2-b8d3-05898c1d4af6 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:31.816982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:31.845859 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0b814-e926-4fb2-b8d3-05898c1d4af6 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:31.879191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff4235-eb02-4940-a3bb-a54c97faff56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:31.879839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff4235-eb02-4940-a3bb-a54c97faff56 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:31.880122 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff4235-eb02-4940-a3bb-a54c97faff56 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:31.880397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff4235-eb02-4940-a3bb-a54c97faff56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:31.881259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff4235-eb02-4940-a3bb-a54c97faff56 None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=ff03ff4de1984e37b2cf45e648908f15, user.id=a220c43e5ca041a5a8435460d287c4d9) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:04:31.881575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff4235-eb02-4940-a3bb-a54c97faff56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:31.883018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff4235-eb02-4940-a3bb-a54c97faff56 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:31.888841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 584/1185] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:04:31 2022] GET /identity/v3/role_assignments?user.id=a220c43e5ca041a5a8435460d287c4d9&scope.project.id=ff03ff4de1984e37b2cf45e648908f15&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:34.059496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2988e64-366d-4ba9-8c88-6a92d1df1108 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:34.060039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2988e64-366d-4ba9-8c88-6a92d1df1108 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:34.060039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2988e64-366d-4ba9-8c88-6a92d1df1108 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:34.061449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 602/1186] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:04:34.068381 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de779e-1a24-4bcb-85e2-343eeb0c7911 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:34.068939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de779e-1a24-4bcb-85e2-343eeb0c7911 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:34.068939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de779e-1a24-4bcb-85e2-343eeb0c7911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:34.090454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-99de779e-1a24-4bcb-85e2-343eeb0c7911 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:04:34.091777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-99de779e-1a24-4bcb-85e2-343eeb0c7911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:04:34.177719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 585/1187] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:04:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:04:34.202156 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2988e64-366d-4ba9-8c88-6a92d1df1108 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:34.238990 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:34.304360 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2988e64-366d-4ba9-8c88-6a92d1df1108 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:34.343534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d342180-a113-45a4-8053-a7c6344e02bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:34.344244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d342180-a113-45a4-8053-a7c6344e02bc None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:34.344529 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d342180-a113-45a4-8053-a7c6344e02bc None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:34.344808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d342180-a113-45a4-8053-a7c6344e02bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:34.355888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d342180-a113-45a4-8053-a7c6344e02bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-3d342180-a113-45a4-8053-a7c6344e02bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:04:34.363735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:04:34.364823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:04:34.365858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:04:34.365858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:04:34.365858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:04:34.365858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:04:34.365858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:04:34.365858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:04:34.365858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:04:34.365858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:04:34.365858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:04:34.365858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:04:34.365858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:04:34.365858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:04:34.365858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 603/1188] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:04:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:04:34.371438 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de779e-1a24-4bcb-85e2-343eeb0c7911 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:34.435784 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:34.471222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de779e-1a24-4bcb-85e2-343eeb0c7911 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:34.505582 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281986c3-4e4f-4292-9372-553fa93d1eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:34.506260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281986c3-4e4f-4292-9372-553fa93d1eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:34.506545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281986c3-4e4f-4292-9372-553fa93d1eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:34.506817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281986c3-4e4f-4292-9372-553fa93d1eb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:34.507575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281986c3-4e4f-4292-9372-553fa93d1eb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:04:34.507882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281986c3-4e4f-4292-9372-553fa93d1eb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:34.509806 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281986c3-4e4f-4292-9372-553fa93d1eb7 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:34.519412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 586/1189] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:04:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:34.525492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d342180-a113-45a4-8053-a7c6344e02bc None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:34.620595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:34.626472 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d342180-a113-45a4-8053-a7c6344e02bc None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:34.664839 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d521df5-0993-4cc3-aaa4-ad3a97e66fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:34.665907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d521df5-0993-4cc3-aaa4-ad3a97e66fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:34.666196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d521df5-0993-4cc3-aaa4-ad3a97e66fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:34.666477 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d521df5-0993-4cc3-aaa4-ad3a97e66fc8 None admin] PATH_INFO: `/v3/users/a220c43e5ca041a5a8435460d287c4d9` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:34.667673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d521df5-0993-4cc3-aaa4-ad3a97e66fc8 None admin] RBAC: Authorizing `identity:get_user(user_id=a220c43e5ca041a5a8435460d287c4d9)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:34.669140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d521df5-0993-4cc3-aaa4-ad3a97e66fc8 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:34.670774 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 604/1190] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:04:34 2022] GET /identity/v3/users/a220c43e5ca041a5a8435460d287c4d9 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:34.676814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281986c3-4e4f-4292-9372-553fa93d1eb7 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:34.774589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281986c3-4e4f-4292-9372-553fa93d1eb7 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:34.799611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:34.807913 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0f266-70eb-4a3a-bb16-b3bf9b3d4f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:34.812675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0f266-70eb-4a3a-bb16-b3bf9b3d4f69 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:34.812675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0f266-70eb-4a3a-bb16-b3bf9b3d4f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:34.812675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0f266-70eb-4a3a-bb16-b3bf9b3d4f69 None admin] PATH_INFO: `/v3/projects/ff03ff4de1984e37b2cf45e648908f15` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:34.812675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0f266-70eb-4a3a-bb16-b3bf9b3d4f69 None admin] RBAC: Authorizing `identity:get_project(project_id=ff03ff4de1984e37b2cf45e648908f15)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:34.812675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0f266-70eb-4a3a-bb16-b3bf9b3d4f69 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:34.813613 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 587/1191] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:04:34 2022] GET /identity/v3/projects/ff03ff4de1984e37b2cf45e648908f15 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:34.819661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d521df5-0993-4cc3-aaa4-ad3a97e66fc8 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:34.918241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d521df5-0993-4cc3-aaa4-ad3a97e66fc8 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:34.951207 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff826f8c-fce6-419a-a166-a122918f7924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:34.952088 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff826f8c-fce6-419a-a166-a122918f7924 None admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:34.952442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff826f8c-fce6-419a-a166-a122918f7924 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:34.952727 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff826f8c-fce6-419a-a166-a122918f7924 None admin] PATH_INFO: `/v3/projects/ff03ff4de1984e37b2cf45e648908f15/users/a220c43e5ca041a5a8435460d287c4d9/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:34.954687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff826f8c-fce6-419a-a166-a122918f7924 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff03ff4de1984e37b2cf45e648908f15, user_id=a220c43e5ca041a5a8435460d287c4d9, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:34.956119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff826f8c-fce6-419a-a166-a122918f7924 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:35.059846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 605/1192] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:04:34 2022] PUT /identity/v3/projects/ff03ff4de1984e37b2cf45e648908f15/users/a220c43e5ca041a5a8435460d287c4d9/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:04:37.174452 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c9085-db1f-47c7-96fb-f755822e4276 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:37.175024 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c9085-db1f-47c7-96fb-f755822e4276 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:37.175024 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c9085-db1f-47c7-96fb-f755822e4276 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:37.176377 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 588/1193] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:04:37.182089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9e6b4-7a34-46fb-ae39-8b5f83a048b5 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:37.182477 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9e6b4-7a34-46fb-ae39-8b5f83a048b5 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:37.182653 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9e6b4-7a34-46fb-ae39-8b5f83a048b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:37.199404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-82c9e6b4-7a34-46fb-ae39-8b5f83a048b5 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:04:37.200589 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-82c9e6b4-7a34-46fb-ae39-8b5f83a048b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:04:37.217778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:37.266459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 606/1194] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:04:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:04:37.290259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c9085-db1f-47c7-96fb-f755822e4276 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:37.389823 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c9085-db1f-47c7-96fb-f755822e4276 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:37.422719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023c443-9c14-4093-996b-bade53a18bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:37.423497 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023c443-9c14-4093-996b-bade53a18bbe None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:37.424029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023c443-9c14-4093-996b-bade53a18bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:37.424301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023c443-9c14-4093-996b-bade53a18bbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:37.435243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023c443-9c14-4093-996b-bade53a18bbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-3023c443-9c14-4093-996b-bade53a18bbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:04:37.442669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:04:37.443721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:04:37.444991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:04:37.444991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:04:37.444991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:04:37.444991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:04:37.444991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:04:37.444991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:04:37.444991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:04:37.444991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:04:37.444991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:04:37.444991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:04:37.444991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:04:37.444991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:04:37.444991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 589/1195] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:04:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:04:37.449925 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9e6b4-7a34-46fb-ae39-8b5f83a048b5 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:37.548203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9e6b4-7a34-46fb-ae39-8b5f83a048b5 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:37.582129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957464a-4e92-43cd-b78c-1fc38561c9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:37.582787 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957464a-4e92-43cd-b78c-1fc38561c9cb None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:37.583067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957464a-4e92-43cd-b78c-1fc38561c9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:37.583340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957464a-4e92-43cd-b78c-1fc38561c9cb None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:37.584090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957464a-4e92-43cd-b78c-1fc38561c9cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:04:37.584392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957464a-4e92-43cd-b78c-1fc38561c9cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:37.585787 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957464a-4e92-43cd-b78c-1fc38561c9cb None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:37.594604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 607/1196] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:04:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:37.600321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023c443-9c14-4093-996b-bade53a18bbe None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:37.655873 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:37.698615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023c443-9c14-4093-996b-bade53a18bbe None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:37.733245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a867647-b932-49f2-9432-df3ded74bc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:37.734099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a867647-b932-49f2-9432-df3ded74bc19 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:37.734381 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a867647-b932-49f2-9432-df3ded74bc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:37.734701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a867647-b932-49f2-9432-df3ded74bc19 None admin] PATH_INFO: `/v3/users/a220c43e5ca041a5a8435460d287c4d9` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:37.735862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a867647-b932-49f2-9432-df3ded74bc19 None admin] RBAC: Authorizing `identity:get_user(user_id=a220c43e5ca041a5a8435460d287c4d9)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:37.737296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a867647-b932-49f2-9432-df3ded74bc19 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:37.738887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 590/1197] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jun 15 10:04:37 2022] GET /identity/v3/users/a220c43e5ca041a5a8435460d287c4d9 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:37.745240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957464a-4e92-43cd-b78c-1fc38561c9cb None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:37.763751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:37.843528 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957464a-4e92-43cd-b78c-1fc38561c9cb None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:37.876479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e88f9e-302a-4abe-94d2-6156d710b22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:37.877147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e88f9e-302a-4abe-94d2-6156d710b22f None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:37.877433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e88f9e-302a-4abe-94d2-6156d710b22f None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:37.877709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e88f9e-302a-4abe-94d2-6156d710b22f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:37.943234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e88f9e-302a-4abe-94d2-6156d710b22f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:37.944688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e88f9e-302a-4abe-94d2-6156d710b22f None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:37.961141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:38.041549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 608/1198] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:04:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:04:38.047455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a867647-b932-49f2-9432-df3ded74bc19 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:38.144848 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a867647-b932-49f2-9432-df3ded74bc19 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:38.168147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:38.178851 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40275212-62c0-4e9b-b76c-25add44f7dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:38.179710 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40275212-62c0-4e9b-b76c-25add44f7dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:38.180025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40275212-62c0-4e9b-b76c-25add44f7dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:38.180316 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40275212-62c0-4e9b-b76c-25add44f7dd1 None admin] PATH_INFO: `/v3/projects/ff03ff4de1984e37b2cf45e648908f15` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:38.181529 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40275212-62c0-4e9b-b76c-25add44f7dd1 None admin] RBAC: Authorizing `identity:get_project(project_id=ff03ff4de1984e37b2cf45e648908f15)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:38.183048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40275212-62c0-4e9b-b76c-25add44f7dd1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:38.184733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 591/1199] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jun 15 10:04:38 2022] GET /identity/v3/projects/ff03ff4de1984e37b2cf45e648908f15 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:38.190796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e88f9e-302a-4abe-94d2-6156d710b22f None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:04:38.288426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e88f9e-302a-4abe-94d2-6156d710b22f None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:04:38.304630 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:38.323209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c148a5-6bf5-4476-80c6-fde273a09b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:04:38.323858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c148a5-6bf5-4476-80c6-fde273a09b24 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:04:38.324150 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c148a5-6bf5-4476-80c6-fde273a09b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:04:38.324426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c148a5-6bf5-4476-80c6-fde273a09b24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:04:38.325298 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c148a5-6bf5-4476-80c6-fde273a09b24 None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=ff03ff4de1984e37b2cf45e648908f15, user.id=a220c43e5ca041a5a8435460d287c4d9) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:04:38.325617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c148a5-6bf5-4476-80c6-fde273a09b24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:04:38.327091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c148a5-6bf5-4476-80c6-fde273a09b24 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:04:38.333038 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 609/1200] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:04:38 2022] GET /identity/v3/role_assignments?user.id=a220c43e5ca041a5a8435460d287c4d9&scope.project.id=ff03ff4de1984e37b2cf45e648908f15&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:04:48.197598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:04:48.334669 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=10 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:05:58.228979 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302a128-0672-42dd-b0d6-f2b8092692a4 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:05:58.229573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302a128-0672-42dd-b0d6-f2b8092692a4 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:05:58.229573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302a128-0672-42dd-b0d6-f2b8092692a4 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:05:58.231070 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 592/1201] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:05:58.236497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc317d5e-c703-438c-a6de-dfdb7d4ee305 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:05:58.237212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc317d5e-c703-438c-a6de-dfdb7d4ee305 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:05:58.237212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc317d5e-c703-438c-a6de-dfdb7d4ee305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:05:58.256470 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-dc317d5e-c703-438c-a6de-dfdb7d4ee305 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:05:58.257717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-dc317d5e-c703-438c-a6de-dfdb7d4ee305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:05:58.324235 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 610/1202] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:05:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:05:58.348792 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302a128-0672-42dd-b0d6-f2b8092692a4 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:05:58.453824 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302a128-0672-42dd-b0d6-f2b8092692a4 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:05:58.488641 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee933d-5b85-465a-903b-a75732751005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:05:58.489410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee933d-5b85-465a-903b-a75732751005 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:05:58.489693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee933d-5b85-465a-903b-a75732751005 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:05:58.489965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee933d-5b85-465a-903b-a75732751005 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:05:58.501558 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee933d-5b85-465a-903b-a75732751005 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:05:58.503022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee933d-5b85-465a-903b-a75732751005 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:05:58.521082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-cbee933d-5b85-465a-903b-a75732751005 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:05:58.521082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:05:58.521082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:05:58.521082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:05:58.521082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:05:58.521082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:05:58.521082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:05:58.521082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:05:58.521082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:05:58.521082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:05:58.521082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:05:58.521082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:05:58.521082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:05:58.521082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:05:58.521082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:05:58.522151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:05:58.526289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:05:58.526289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:05:58.526289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:05:58.526289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:05:58.526289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 593/1203] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:05:58.528581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc317d5e-c703-438c-a6de-dfdb7d4ee305 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:05:58.629987 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc317d5e-c703-438c-a6de-dfdb7d4ee305 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:05:58.659239 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:05:58.665017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5c960-502a-4df0-97da-80835230ec2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:05:58.665854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5c960-502a-4df0-97da-80835230ec2c None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:05:58.666139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5c960-502a-4df0-97da-80835230ec2c None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:05:58.666416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5c960-502a-4df0-97da-80835230ec2c None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:05:58.667144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5c960-502a-4df0-97da-80835230ec2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:05:58.667450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5c960-502a-4df0-97da-80835230ec2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:05:58.668861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f5c960-502a-4df0-97da-80835230ec2c None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:05:58.681030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 611/1204] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:05:58.687124 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee933d-5b85-465a-903b-a75732751005 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:05:58.741646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:05:58.789730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee933d-5b85-465a-903b-a75732751005 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:05:58.824616 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28369ff-993e-41d7-9116-f97f2d0c859c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:05:58.825324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28369ff-993e-41d7-9116-f97f2d0c859c None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:05:58.840120 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28369ff-993e-41d7-9116-f97f2d0c859c None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:05:58.840120 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28369ff-993e-41d7-9116-f97f2d0c859c None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:05:58.840120 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28369ff-993e-41d7-9116-f97f2d0c859c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:05:58.840120 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28369ff-993e-41d7-9116-f97f2d0c859c None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:05:58.953292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 594/1205] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jun 15 10:05:58 2022] POST /identity/v3/users => generated 251 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:06:01.129665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad84df5-488d-4a13-9512-714255db1c52 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:01.130259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad84df5-488d-4a13-9512-714255db1c52 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:01.130259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad84df5-488d-4a13-9512-714255db1c52 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:01.131627 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 612/1206] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:06:01.136716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f12420-fa5d-449b-9dc4-c68c1a774473 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:01.137176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f12420-fa5d-449b-9dc4-c68c1a774473 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:01.137320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f12420-fa5d-449b-9dc4-c68c1a774473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:01.153869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a0f12420-fa5d-449b-9dc4-c68c1a774473 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:06:01.155059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a0f12420-fa5d-449b-9dc4-c68c1a774473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:06:01.224099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 595/1207] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:06:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:06:01.248162 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad84df5-488d-4a13-9512-714255db1c52 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:01.351523 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad84df5-488d-4a13-9512-714255db1c52 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:01.387700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcddd5d-7983-47f9-a3f4-f7a727410723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:01.388428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcddd5d-7983-47f9-a3f4-f7a727410723 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:01.388713 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcddd5d-7983-47f9-a3f4-f7a727410723 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:01.389013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcddd5d-7983-47f9-a3f4-f7a727410723 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:01.401642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcddd5d-7983-47f9-a3f4-f7a727410723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-7dcddd5d-7983-47f9-a3f4-f7a727410723 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:06:01.411314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:01.412460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:01.413519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:01.413519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:01.413519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:01.413519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:01.413519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:06:01.413519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:06:01.413519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:06:01.413519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:06:01.413519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:06:01.413519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:06:01.413519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:06:01.413519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:01.413519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 613/1208] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:06:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:01.418624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f12420-fa5d-449b-9dc4-c68c1a774473 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:01.458943 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:01.518317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f12420-fa5d-449b-9dc4-c68c1a774473 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:01.553954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e77aff-3069-469f-bb8a-269d3b01c1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:01.554729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e77aff-3069-469f-bb8a-269d3b01c1ba None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:01.555017 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e77aff-3069-469f-bb8a-269d3b01c1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:01.555298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e77aff-3069-469f-bb8a-269d3b01c1ba None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:01.556057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e77aff-3069-469f-bb8a-269d3b01c1ba None admin] RBAC: Adding query filter params (name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:01.556372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e77aff-3069-469f-bb8a-269d3b01c1ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:01.557772 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e77aff-3069-469f-bb8a-269d3b01c1ba None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:01.566789 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 596/1209] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:06:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:01.572615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcddd5d-7983-47f9-a3f4-f7a727410723 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:01.626753 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:01.680414 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcddd5d-7983-47f9-a3f4-f7a727410723 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:01.718535 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88c2a6-d57d-42f5-9ab9-f42c45996d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:01.719835 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88c2a6-d57d-42f5-9ab9-f42c45996d64 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:01.720243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88c2a6-d57d-42f5-9ab9-f42c45996d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:01.720705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88c2a6-d57d-42f5-9ab9-f42c45996d64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:01.731351 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88c2a6-d57d-42f5-9ab9-f42c45996d64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:01.732881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88c2a6-d57d-42f5-9ab9-f42c45996d64 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:01.753027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-ac88c2a6-d57d-42f5-9ab9-f42c45996d64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:01.753027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:01.753027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:01.753027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:01.753027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:01.753027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:01.753027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:01.753027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:01.753027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:01.753027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:01.753027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:01.753027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:01.753027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:06:01.753027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:06:01.753027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:01.754122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:01.755249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:06:01.755249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:06:01.755249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:01.755249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:01.755249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 614/1210] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:01.760277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e77aff-3069-469f-bb8a-269d3b01c1ba None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:01.863043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:01.866120 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e77aff-3069-469f-bb8a-269d3b01c1ba None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:01.899591 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4559f8-88f8-47d5-be6b-2aa6adb76004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:01.900260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4559f8-88f8-47d5-be6b-2aa6adb76004 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:01.900547 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4559f8-88f8-47d5-be6b-2aa6adb76004 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:01.900825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4559f8-88f8-47d5-be6b-2aa6adb76004 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:01.901596 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4559f8-88f8-47d5-be6b-2aa6adb76004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:01.901900 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4559f8-88f8-47d5-be6b-2aa6adb76004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:01.903396 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4559f8-88f8-47d5-be6b-2aa6adb76004 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:01.916453 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 597/1211] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:01.922744 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88c2a6-d57d-42f5-9ab9-f42c45996d64 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:02.023893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88c2a6-d57d-42f5-9ab9-f42c45996d64 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:02.025467 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:02.058384 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2e1ae-33e5-48eb-b041-913c1ad74c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:02.059317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2e1ae-33e5-48eb-b041-913c1ad74c38 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:02.059606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2e1ae-33e5-48eb-b041-913c1ad74c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:02.059883 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2e1ae-33e5-48eb-b041-913c1ad74c38 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:02.067268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2e1ae-33e5-48eb-b041-913c1ad74c38 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:02.068728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2e1ae-33e5-48eb-b041-913c1ad74c38 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-1cb2e1ae-33e5-48eb-b041-913c1ad74c38 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:06:02.085017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:02.086245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:02.087497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:02.087497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:02.087497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:06:02.087497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:06:02.087497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:06:02.087497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:06:02.087497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:06:02.087497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:06:02.087497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:06:02.087497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:02.087497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 615/1212] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:02.092250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4559f8-88f8-47d5-be6b-2aa6adb76004 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:02.196165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4559f8-88f8-47d5-be6b-2aa6adb76004 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:02.229468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8efa0d-f22e-453b-b122-7132a82c32c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:02.230128 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8efa0d-f22e-453b-b122-7132a82c32c5 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:02.230412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8efa0d-f22e-453b-b122-7132a82c32c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:02.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8efa0d-f22e-453b-b122-7132a82c32c5 None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:02.231570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8efa0d-f22e-453b-b122-7132a82c32c5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:02.231879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8efa0d-f22e-453b-b122-7132a82c32c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:02.233337 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8efa0d-f22e-453b-b122-7132a82c32c5 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:02.251981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 598/1213] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Jun 15 10:06:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:02.257945 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2e1ae-33e5-48eb-b041-913c1ad74c38 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:02.361813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2e1ae-33e5-48eb-b041-913c1ad74c38 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:02.399311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44924a2e-27d7-4fce-93cb-2685314d6f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:02.399968 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44924a2e-27d7-4fce-93cb-2685314d6f03 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:02.400252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44924a2e-27d7-4fce-93cb-2685314d6f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:02.400532 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44924a2e-27d7-4fce-93cb-2685314d6f03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:02.431514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:02.468555 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44924a2e-27d7-4fce-93cb-2685314d6f03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:02.470824 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44924a2e-27d7-4fce-93cb-2685314d6f03 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:02.571273 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 616/1214] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:06:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:06:02.577237 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8efa0d-f22e-453b-b122-7132a82c32c5 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:02.597517 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:02.678482 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8efa0d-f22e-453b-b122-7132a82c32c5 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:02.713134 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9590bc-7247-4bc8-9c7e-0829d60cc89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:02.713928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9590bc-7247-4bc8-9c7e-0829d60cc89d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:02.714209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9590bc-7247-4bc8-9c7e-0829d60cc89d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:02.714485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9590bc-7247-4bc8-9c7e-0829d60cc89d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:02.725083 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9590bc-7247-4bc8-9c7e-0829d60cc89d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:02.726519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9590bc-7247-4bc8-9c7e-0829d60cc89d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:02.744340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-6a9590bc-7247-4bc8-9c7e-0829d60cc89d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:02.744340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:02.744340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:02.744340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:02.744340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:02.744340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:02.744340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:02.744340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:02.744340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:02.744340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:02.744340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:02.744340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:02.744340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:06:02.744340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:06:02.744340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:02.745551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:02.746655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:06:02.746655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:06:02.746655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:02.746655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:06:02.746655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 599/1215] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:02.751627 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44924a2e-27d7-4fce-93cb-2685314d6f03 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:02.813626 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:02.853295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44924a2e-27d7-4fce-93cb-2685314d6f03 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:02.887216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e2da1-93b4-463a-9cd9-60aacba39507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:02.887873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e2da1-93b4-463a-9cd9-60aacba39507 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:02.888159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e2da1-93b4-463a-9cd9-60aacba39507 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:02.888439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e2da1-93b4-463a-9cd9-60aacba39507 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:02.889196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e2da1-93b4-463a-9cd9-60aacba39507 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:02.889507 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e2da1-93b4-463a-9cd9-60aacba39507 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:02.891453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e2da1-93b4-463a-9cd9-60aacba39507 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:02.903047 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 617/1216] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:02.910076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9590bc-7247-4bc8-9c7e-0829d60cc89d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:02.981154 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:03.010608 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9590bc-7247-4bc8-9c7e-0829d60cc89d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:03.045577 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2244d9-ab35-4cf2-a79c-e56104c3fd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:03.046362 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2244d9-ab35-4cf2-a79c-e56104c3fd61 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:03.046642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2244d9-ab35-4cf2-a79c-e56104c3fd61 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:03.047036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2244d9-ab35-4cf2-a79c-e56104c3fd61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:03.054025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2244d9-ab35-4cf2-a79c-e56104c3fd61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:03.055538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2244d9-ab35-4cf2-a79c-e56104c3fd61 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:03.070954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-1a2244d9-ab35-4cf2-a79c-e56104c3fd61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:06:03.070954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:03.070954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:03.070954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:03.070954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:03.070954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:03.070954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:03.070954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:03.070954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:03.070954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:03.070954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:03.070954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:03.070954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:06:03.070954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:06:03.070954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:03.072011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:03.073177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:06:03.073177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:06:03.073177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:06:03.073177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:06:03.073177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:06:03.073177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:06:03.073177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:06:03.073177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:06:03.073177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 600/1217] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:06:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:03.078744 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e2da1-93b4-463a-9cd9-60aacba39507 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:03.178061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e2da1-93b4-463a-9cd9-60aacba39507 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:03.200780 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:03.212735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ae667a-5044-4001-a976-28d9e94e8f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:03.213445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae667a-5044-4001-a976-28d9e94e8f99 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:03.213743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae667a-5044-4001-a976-28d9e94e8f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:03.214024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae667a-5044-4001-a976-28d9e94e8f99 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:03.214824 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae667a-5044-4001-a976-28d9e94e8f99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:03.215138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae667a-5044-4001-a976-28d9e94e8f99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:03.216582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ae667a-5044-4001-a976-28d9e94e8f99 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:03.229639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 618/1218] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:06:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:03.236692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2244d9-ab35-4cf2-a79c-e56104c3fd61 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:03.339598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2244d9-ab35-4cf2-a79c-e56104c3fd61 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:03.367589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:03.373160 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2abcc4b-dd7c-4dbc-88c6-dab90857c6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:03.373846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abcc4b-dd7c-4dbc-88c6-dab90857c6bc None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:03.374212 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abcc4b-dd7c-4dbc-88c6-dab90857c6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:03.374446 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2abcc4b-dd7c-4dbc-88c6-dab90857c6bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:03.375997 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2abcc4b-dd7c-4dbc-88c6-dab90857c6bc None admin] RBAC: Adding query filter params (role.id=2c276d442f7f4dd59ae7fc3a03943716, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=12899a7a59b54e3199b277ff94e30d42) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:03.376319 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2abcc4b-dd7c-4dbc-88c6-dab90857c6bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:03.377774 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2abcc4b-dd7c-4dbc-88c6-dab90857c6bc None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:03.383268 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 601/1219] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:06:03 2022] GET /identity/v3/role_assignments?user.id=12899a7a59b54e3199b277ff94e30d42&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=2c276d442f7f4dd59ae7fc3a03943716 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:05.542442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304d247-36ea-498d-86b9-419c8b12f055 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:05.542959 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304d247-36ea-498d-86b9-419c8b12f055 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:05.543080 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304d247-36ea-498d-86b9-419c8b12f055 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:05.544466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 619/1220] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:06:05.550718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71960d-2151-44fe-88bc-790a8a732e38 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:05.551151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71960d-2151-44fe-88bc-790a8a732e38 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:05.551289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71960d-2151-44fe-88bc-790a8a732e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:05.567872 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-ad71960d-2151-44fe-88bc-790a8a732e38 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:06:05.569076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ad71960d-2151-44fe-88bc-790a8a732e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:06:05.635098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 602/1221] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:06:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:06:05.658268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304d247-36ea-498d-86b9-419c8b12f055 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:05.761835 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304d247-36ea-498d-86b9-419c8b12f055 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:05.796769 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21877d78-7747-4184-a994-7e8bd6aee5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:05.797598 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21877d78-7747-4184-a994-7e8bd6aee5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:05.797973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21877d78-7747-4184-a994-7e8bd6aee5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:05.798254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21877d78-7747-4184-a994-7e8bd6aee5b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:05.808771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21877d78-7747-4184-a994-7e8bd6aee5b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-21877d78-7747-4184-a994-7e8bd6aee5b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:06:05.816828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:05.817953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:05.819552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:05.819552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:05.819552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:05.819552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:05.819552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:06:05.819552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:06:05.819552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:06:05.819552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:06:05.819552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:06:05.819552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:06:05.819552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:06:05.819552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:05.819552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 620/1222] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:06:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:05.824814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71960d-2151-44fe-88bc-790a8a732e38 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:05.931360 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71960d-2151-44fe-88bc-790a8a732e38 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:05.955437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:05.965752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a3a62-4fe7-45f1-948a-090023b4d077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:05.966455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a3a62-4fe7-45f1-948a-090023b4d077 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:05.966743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a3a62-4fe7-45f1-948a-090023b4d077 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:05.967017 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a3a62-4fe7-45f1-948a-090023b4d077 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:05.967775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a3a62-4fe7-45f1-948a-090023b4d077 None admin] RBAC: Adding query filter params (name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:05.968085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a3a62-4fe7-45f1-948a-090023b4d077 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:05.970094 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a3a62-4fe7-45f1-948a-090023b4d077 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:05.978259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 603/1223] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:06:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:05.984213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21877d78-7747-4184-a994-7e8bd6aee5b3 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:06.028977 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:06.086526 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21877d78-7747-4184-a994-7e8bd6aee5b3 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:06.120702 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059d2a1-0758-45f6-aac5-cade1ac6bd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:06.121564 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059d2a1-0758-45f6-aac5-cade1ac6bd46 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:06.121877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059d2a1-0758-45f6-aac5-cade1ac6bd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:06.133224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059d2a1-0758-45f6-aac5-cade1ac6bd46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:06.133224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059d2a1-0758-45f6-aac5-cade1ac6bd46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:06.134412 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059d2a1-0758-45f6-aac5-cade1ac6bd46 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:06.152043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-d059d2a1-0758-45f6-aac5-cade1ac6bd46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:06.152043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:06.152043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:06.152043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:06.152043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:06.152043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:06.152043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:06.152043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:06.152043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:06.152043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:06.152043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:06.152043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:06.152043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:06:06.152043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:06:06.152043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:06.152984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:06.154011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:06:06.154011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:06:06.154011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:06.154011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:06.154011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 621/1224] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:06.159719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a3a62-4fe7-45f1-948a-090023b4d077 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:06.260536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a3a62-4fe7-45f1-948a-090023b4d077 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:06.293915 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99478d12-306c-4e54-91f1-03b2b1824f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:06.294576 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99478d12-306c-4e54-91f1-03b2b1824f32 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:06.294863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99478d12-306c-4e54-91f1-03b2b1824f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:06.295140 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99478d12-306c-4e54-91f1-03b2b1824f32 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:06.295868 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99478d12-306c-4e54-91f1-03b2b1824f32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:06.296177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99478d12-306c-4e54-91f1-03b2b1824f32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:06.298023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99478d12-306c-4e54-91f1-03b2b1824f32 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:06.310204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 604/1225] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:06.316380 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059d2a1-0758-45f6-aac5-cade1ac6bd46 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:06.409943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:06.416566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059d2a1-0758-45f6-aac5-cade1ac6bd46 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:06.450770 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2009060-b9ff-4c5a-89eb-a0b3ae11cc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:06.452892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2009060-b9ff-4c5a-89eb-a0b3ae11cc7e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:06.452892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2009060-b9ff-4c5a-89eb-a0b3ae11cc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:06.452892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2009060-b9ff-4c5a-89eb-a0b3ae11cc7e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:06.460806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2009060-b9ff-4c5a-89eb-a0b3ae11cc7e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:06.462283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2009060-b9ff-4c5a-89eb-a0b3ae11cc7e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-f2009060-b9ff-4c5a-89eb-a0b3ae11cc7e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:06:06.476981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:06.478099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:06.479142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:06.479142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:06.479142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:06:06.479142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:06:06.479142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:06:06.479142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:06:06.479142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:06:06.479142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:06:06.479142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:06:06.479142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:06.479142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 622/1226] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:06.484105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99478d12-306c-4e54-91f1-03b2b1824f32 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:06.531557 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:06.585456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99478d12-306c-4e54-91f1-03b2b1824f32 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:06.620137 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864cdb2-cbcb-412b-98b8-723b9cf80649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:06.620818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864cdb2-cbcb-412b-98b8-723b9cf80649 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:06.621132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864cdb2-cbcb-412b-98b8-723b9cf80649 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:06.621405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864cdb2-cbcb-412b-98b8-723b9cf80649 None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:06.622255 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864cdb2-cbcb-412b-98b8-723b9cf80649 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:06.622559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864cdb2-cbcb-412b-98b8-723b9cf80649 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:06.624039 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864cdb2-cbcb-412b-98b8-723b9cf80649 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:06.642424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 605/1227] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Jun 15 10:06:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:06.648356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2009060-b9ff-4c5a-89eb-a0b3ae11cc7e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:06.758268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2009060-b9ff-4c5a-89eb-a0b3ae11cc7e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:06.792161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0cb20a-9c67-440d-a371-6009c435d34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:06.793235 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0cb20a-9c67-440d-a371-6009c435d34a None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:06.793683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0cb20a-9c67-440d-a371-6009c435d34a None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:06.793967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0cb20a-9c67-440d-a371-6009c435d34a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:06.805903 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0cb20a-9c67-440d-a371-6009c435d34a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:06.807361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0cb20a-9c67-440d-a371-6009c435d34a None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:06.828145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-ce0cb20a-9c67-440d-a371-6009c435d34a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:06.828145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:06.828145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:06.828145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:06.828145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:06.828145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:06.828145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:06.828145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:06.828145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:06.828145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:06.828145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:06.828145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:06.828145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:06:06.828145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:06:06.828145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:06.829818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:06.830860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:06:06.830860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:06:06.830860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:06.830860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:06.830860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 623/1228] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:06.835612 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864cdb2-cbcb-412b-98b8-723b9cf80649 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:06.938899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864cdb2-cbcb-412b-98b8-723b9cf80649 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:06.941241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:06.974191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d61a3c-5f5c-4d87-b599-3e5fc024bbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:06.974860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d61a3c-5f5c-4d87-b599-3e5fc024bbaf None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:06.975144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d61a3c-5f5c-4d87-b599-3e5fc024bbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:06.975422 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d61a3c-5f5c-4d87-b599-3e5fc024bbaf None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:06.976163 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d61a3c-5f5c-4d87-b599-3e5fc024bbaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:06.976476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d61a3c-5f5c-4d87-b599-3e5fc024bbaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:06.977919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d61a3c-5f5c-4d87-b599-3e5fc024bbaf None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:06.990118 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 606/1229] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:06.995938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0cb20a-9c67-440d-a371-6009c435d34a None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:07.006462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:07.097978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0cb20a-9c67-440d-a371-6009c435d34a None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:07.131378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48f5e0-8a77-45ac-9c3f-7499f17b957d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:07.132182 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48f5e0-8a77-45ac-9c3f-7499f17b957d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:07.132469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48f5e0-8a77-45ac-9c3f-7499f17b957d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:07.132745 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48f5e0-8a77-45ac-9c3f-7499f17b957d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:07.139492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48f5e0-8a77-45ac-9c3f-7499f17b957d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:07.141103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48f5e0-8a77-45ac-9c3f-7499f17b957d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:07.155868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-ff48f5e0-8a77-45ac-9c3f-7499f17b957d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:06:07.155868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:07.155868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:07.155868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:07.155868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:07.155868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:07.155868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:07.155868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:07.155868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:07.155868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:07.155868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:07.155868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:07.155868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:06:07.155868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:06:07.155868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:07.157594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:07.158983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:06:07.158983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:06:07.158983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:06:07.158983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:06:07.158983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:06:07.158983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:06:07.158983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:06:07.158983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:07.158983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 624/1230] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:06:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:07.164145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d61a3c-5f5c-4d87-b599-3e5fc024bbaf None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:07.267394 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d61a3c-5f5c-4d87-b599-3e5fc024bbaf None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:07.301889 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde96e8-32f2-4951-8dfe-b9787e015f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:07.302557 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde96e8-32f2-4951-8dfe-b9787e015f16 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:07.302841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde96e8-32f2-4951-8dfe-b9787e015f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:07.303117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde96e8-32f2-4951-8dfe-b9787e015f16 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:07.303949 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde96e8-32f2-4951-8dfe-b9787e015f16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:07.304266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde96e8-32f2-4951-8dfe-b9787e015f16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:07.305725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde96e8-32f2-4951-8dfe-b9787e015f16 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:07.318690 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 607/1231] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:06:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:07.325063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48f5e0-8a77-45ac-9c3f-7499f17b957d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:07.396925 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:07.434123 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48f5e0-8a77-45ac-9c3f-7499f17b957d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:07.468478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffc80e-adb2-4c4c-977c-ccddcfe087d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:07.470048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffc80e-adb2-4c4c-977c-ccddcfe087d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:07.470409 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffc80e-adb2-4c4c-977c-ccddcfe087d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:07.470699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffc80e-adb2-4c4c-977c-ccddcfe087d6 None admin] PATH_INFO: `/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/12899a7a59b54e3199b277ff94e30d42/roles/2c276d442f7f4dd59ae7fc3a03943716` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:07.486304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ffc80e-adb2-4c4c-977c-ccddcfe087d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=db47f1bb607c4d1a9c346fdc80ecadfa, user_id=12899a7a59b54e3199b277ff94e30d42, role_id=2c276d442f7f4dd59ae7fc3a03943716)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:07.490520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ffc80e-adb2-4c4c-977c-ccddcfe087d6 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:07.585980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 625/1232] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:06:07 2022] PUT /identity/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/12899a7a59b54e3199b277ff94e30d42/roles/2c276d442f7f4dd59ae7fc3a03943716 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:06:09.750012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051ac74-dad8-4019-82fd-7e02337aa93a None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:09.750630 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051ac74-dad8-4019-82fd-7e02337aa93a None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:09.750630 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051ac74-dad8-4019-82fd-7e02337aa93a None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:09.752425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 608/1233] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:06:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:06:09.758246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280dab7-d1e8-4dc7-8df7-6be8d1a24153 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:09.758775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280dab7-d1e8-4dc7-8df7-6be8d1a24153 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:09.758775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280dab7-d1e8-4dc7-8df7-6be8d1a24153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:09.776360 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-e280dab7-d1e8-4dc7-8df7-6be8d1a24153 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:06:09.777636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e280dab7-d1e8-4dc7-8df7-6be8d1a24153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:06:09.850650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 626/1234] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:06:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:06:09.875096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051ac74-dad8-4019-82fd-7e02337aa93a None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:09.904745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:09.988172 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051ac74-dad8-4019-82fd-7e02337aa93a None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:10.025581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f778e0-455e-4220-a84c-6c98e4177382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:10.026394 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f778e0-455e-4220-a84c-6c98e4177382 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:10.026828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f778e0-455e-4220-a84c-6c98e4177382 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:10.027110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f778e0-455e-4220-a84c-6c98e4177382 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:10.038480 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f778e0-455e-4220-a84c-6c98e4177382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-21f778e0-455e-4220-a84c-6c98e4177382 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:06:10.046302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:10.048034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:10.049623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:10.049623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:10.049623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:10.049623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:10.049623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:06:10.049623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:06:10.049623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:06:10.049623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:06:10.049623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:06:10.049623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:06:10.049623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 15 10:06:10.049623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:06:10.049623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 609/1235] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jun 15 10:06:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:10.057235 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280dab7-d1e8-4dc7-8df7-6be8d1a24153 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:10.104157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:10.158817 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280dab7-d1e8-4dc7-8df7-6be8d1a24153 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:10.194338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d529ae0-20af-46f3-bee7-878395fd77eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:10.195004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d529ae0-20af-46f3-bee7-878395fd77eb None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:10.195292 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d529ae0-20af-46f3-bee7-878395fd77eb None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:10.195571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d529ae0-20af-46f3-bee7-878395fd77eb None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:10.196324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d529ae0-20af-46f3-bee7-878395fd77eb None admin] RBAC: Adding query filter params (name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:10.196636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d529ae0-20af-46f3-bee7-878395fd77eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:10.198051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d529ae0-20af-46f3-bee7-878395fd77eb None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:10.207403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 627/1236] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jun 15 10:06:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:10.213198 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f778e0-455e-4220-a84c-6c98e4177382 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:10.301363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:10.318531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f778e0-455e-4220-a84c-6c98e4177382 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:10.353220 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364711dd-537a-4673-ba0a-d8f51a0ad227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:10.354016 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364711dd-537a-4673-ba0a-d8f51a0ad227 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:10.354348 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364711dd-537a-4673-ba0a-d8f51a0ad227 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:10.354578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364711dd-537a-4673-ba0a-d8f51a0ad227 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:10.365668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364711dd-537a-4673-ba0a-d8f51a0ad227 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:10.367222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364711dd-537a-4673-ba0a-d8f51a0ad227 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:10.384551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-364711dd-537a-4673-ba0a-d8f51a0ad227 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:10.384551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:10.384551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:10.384551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:10.384551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:10.384551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:10.384551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:10.384551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:10.384551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:10.384551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:10.384551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:10.384551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:10.384551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:06:10.384551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:06:10.384551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:10.385840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:10.386719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:06:10.386719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:06:10.386719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:10.386719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:06:10.386719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 610/1237] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:10.392443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d529ae0-20af-46f3-bee7-878395fd77eb None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:10.494733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d529ae0-20af-46f3-bee7-878395fd77eb None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:10.505075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:10.536914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a8574-5319-4249-b5fa-d7ac1beb8f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:10.537620 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a8574-5319-4249-b5fa-d7ac1beb8f49 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:10.537899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a8574-5319-4249-b5fa-d7ac1beb8f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:10.538260 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a8574-5319-4249-b5fa-d7ac1beb8f49 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:10.538993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a8574-5319-4249-b5fa-d7ac1beb8f49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:10.539300 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a8574-5319-4249-b5fa-d7ac1beb8f49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:10.540699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a8574-5319-4249-b5fa-d7ac1beb8f49 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:10.553694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 628/1238] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:06:10.559641 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364711dd-537a-4673-ba0a-d8f51a0ad227 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:10.672696 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364711dd-537a-4673-ba0a-d8f51a0ad227 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:10.708910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8831c62-0aae-4b86-85c9-b69fcf1ca695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:10.709746 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8831c62-0aae-4b86-85c9-b69fcf1ca695 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:10.710038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8831c62-0aae-4b86-85c9-b69fcf1ca695 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:10.710317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8831c62-0aae-4b86-85c9-b69fcf1ca695 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:10.713311 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:10.718638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8831c62-0aae-4b86-85c9-b69fcf1ca695 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:10.720675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8831c62-0aae-4b86-85c9-b69fcf1ca695 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-d8831c62-0aae-4b86-85c9-b69fcf1ca695 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:06:10.736434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:10.737649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:10.738605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:10.738605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:10.738605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:06:10.738605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:06:10.738605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:06:10.738605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:06:10.738605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:06:10.738605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:06:10.738605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:06:10.738605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:06:10.738605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 611/1239] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:10.744080 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a8574-5319-4249-b5fa-d7ac1beb8f49 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:10.849512 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a8574-5319-4249-b5fa-d7ac1beb8f49 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:10.891656 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a50ebb-03be-4e85-9cee-afc83c674dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:10.892562 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a50ebb-03be-4e85-9cee-afc83c674dde None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:10.893034 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a50ebb-03be-4e85-9cee-afc83c674dde None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:10.893404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a50ebb-03be-4e85-9cee-afc83c674dde None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:10.894387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a50ebb-03be-4e85-9cee-afc83c674dde None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:10.894789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a50ebb-03be-4e85-9cee-afc83c674dde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:10.897625 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a50ebb-03be-4e85-9cee-afc83c674dde None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:10.916303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 629/1240] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Jun 15 10:06:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:10.922454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8831c62-0aae-4b86-85c9-b69fcf1ca695 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:11.025395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8831c62-0aae-4b86-85c9-b69fcf1ca695 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:11.058427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaf6b1-4246-4d08-a57d-b1d1215fe3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:11.059071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaf6b1-4246-4d08-a57d-b1d1215fe3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:11.059355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaf6b1-4246-4d08-a57d-b1d1215fe3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:11.059635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeaf6b1-4246-4d08-a57d-b1d1215fe3d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:11.119635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:11.126256 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeaf6b1-4246-4d08-a57d-b1d1215fe3d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:11.127794 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeaf6b1-4246-4d08-a57d-b1d1215fe3d8 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:11.224857 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 612/1241] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:06:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:06:11.230846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a50ebb-03be-4e85-9cee-afc83c674dde None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:11.258951 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:11.334309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a50ebb-03be-4e85-9cee-afc83c674dde None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:11.369179 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566883a-8a3e-41a0-93ad-1aeafb74abfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:11.369974 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566883a-8a3e-41a0-93ad-1aeafb74abfb None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:11.370260 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566883a-8a3e-41a0-93ad-1aeafb74abfb None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:11.370540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566883a-8a3e-41a0-93ad-1aeafb74abfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:11.381050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566883a-8a3e-41a0-93ad-1aeafb74abfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:11.382496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566883a-8a3e-41a0-93ad-1aeafb74abfb None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:11.402167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-3566883a-8a3e-41a0-93ad-1aeafb74abfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:11.402167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:11.402167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:11.402167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:11.402167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:11.402167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:11.402167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:11.402167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:11.402167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:11.402167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:11.402167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:11.402167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:11.402167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:06:11.402167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:06:11.402167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:11.403291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:11.404392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:06:11.404392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:06:11.404392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:11.404392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:11.404392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 630/1242] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:11.409337 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaf6b1-4246-4d08-a57d-b1d1215fe3d8 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:11.504614 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:11.510261 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeaf6b1-4246-4d08-a57d-b1d1215fe3d8 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:11.544387 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965033ec-d72c-4862-bcbf-22ce79698b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:11.545105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965033ec-d72c-4862-bcbf-22ce79698b85 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:11.545395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965033ec-d72c-4862-bcbf-22ce79698b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:11.545674 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965033ec-d72c-4862-bcbf-22ce79698b85 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:11.546445 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965033ec-d72c-4862-bcbf-22ce79698b85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:11.546759 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965033ec-d72c-4862-bcbf-22ce79698b85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:11.548186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965033ec-d72c-4862-bcbf-22ce79698b85 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:11.560152 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 613/1243] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:11.566808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566883a-8a3e-41a0-93ad-1aeafb74abfb None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:11.650090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:11.671296 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566883a-8a3e-41a0-93ad-1aeafb74abfb None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:11.723103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e479a-6479-4780-b89c-e6ec9aac881d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:11.723910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e479a-6479-4780-b89c-e6ec9aac881d None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:11.724187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e479a-6479-4780-b89c-e6ec9aac881d None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:11.724475 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e479a-6479-4780-b89c-e6ec9aac881d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:11.731701 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e479a-6479-4780-b89c-e6ec9aac881d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:11.733185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e479a-6479-4780-b89c-e6ec9aac881d None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:11.748751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-d52e479a-6479-4780-b89c-e6ec9aac881d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:06:11.748751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:11.748751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:11.748751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:11.748751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:11.748751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:11.748751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:11.748751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:11.748751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:11.748751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:11.748751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:11.748751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:11.748751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:06:11.748751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:06:11.748751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:11.753089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:11.754004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:06:11.754004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:06:11.754004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:06:11.754004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:06:11.754004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:06:11.754004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:06:11.754004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:06:11.754004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:11.754004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 631/1244] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:06:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:11.757084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965033ec-d72c-4862-bcbf-22ce79698b85 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:11.865233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965033ec-d72c-4862-bcbf-22ce79698b85 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:11.898726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da5a39-64ec-418f-8487-bf49b53b3d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:11.899378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da5a39-64ec-418f-8487-bf49b53b3d72 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:11.899660 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da5a39-64ec-418f-8487-bf49b53b3d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:11.899939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da5a39-64ec-418f-8487-bf49b53b3d72 None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:11.900776 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da5a39-64ec-418f-8487-bf49b53b3d72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:11.901062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da5a39-64ec-418f-8487-bf49b53b3d72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:11.903445 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da5a39-64ec-418f-8487-bf49b53b3d72 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:11.915865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 614/1245] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:06:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:11.921986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e479a-6479-4780-b89c-e6ec9aac881d None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:12.026489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e479a-6479-4780-b89c-e6ec9aac881d None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:12.063591 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf149e7-8935-47a7-82f9-ec82fce576f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:12.064912 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf149e7-8935-47a7-82f9-ec82fce576f9 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:12.065221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf149e7-8935-47a7-82f9-ec82fce576f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:12.067539 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf149e7-8935-47a7-82f9-ec82fce576f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:12.067539 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf149e7-8935-47a7-82f9-ec82fce576f9 None admin] RBAC: Adding query filter params (role.id=2c276d442f7f4dd59ae7fc3a03943716, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=12899a7a59b54e3199b277ff94e30d42) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:12.067539 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf149e7-8935-47a7-82f9-ec82fce576f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:12.068151 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf149e7-8935-47a7-82f9-ec82fce576f9 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:12.074480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 632/1246] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:06:11 2022] GET /identity/v3/role_assignments?user.id=12899a7a59b54e3199b277ff94e30d42&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=2c276d442f7f4dd59ae7fc3a03943716 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:14.288991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5ea13-8cc3-4ce3-adca-74b24cd68311 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:14.289550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5ea13-8cc3-4ce3-adca-74b24cd68311 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:14.289550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5ea13-8cc3-4ce3-adca-74b24cd68311 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:14.290913 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 615/1247] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:06:14.296090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2886e-bd36-4e6c-bd28-0f387fb8b381 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:14.299307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2886e-bd36-4e6c-bd28-0f387fb8b381 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:14.299307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2886e-bd36-4e6c-bd28-0f387fb8b381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:14.303503 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:14.316397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-1fb2886e-bd36-4e6c-bd28-0f387fb8b381 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:06:14.317630 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-1fb2886e-bd36-4e6c-bd28-0f387fb8b381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:06:14.384828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 633/1248] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:06:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:06:14.410966 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5ea13-8cc3-4ce3-adca-74b24cd68311 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:14.430627 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:14.515633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5ea13-8cc3-4ce3-adca-74b24cd68311 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:14.559361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049d496-09df-4c75-a204-0bda9e105cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:14.560193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049d496-09df-4c75-a204-0bda9e105cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:14.560471 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049d496-09df-4c75-a204-0bda9e105cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:14.560743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049d496-09df-4c75-a204-0bda9e105cd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:14.573379 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049d496-09df-4c75-a204-0bda9e105cd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-2049d496-09df-4c75-a204-0bda9e105cd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:06:14.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:14.583797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:14.584678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:14.584678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:14.584678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:14.584678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:14.584678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:06:14.584678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:06:14.584678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:06:14.584678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:06:14.584678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:06:14.584678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:06:14.584678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:06:14.584678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:06:14.584678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 616/1249] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:06:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:14.589580 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2886e-bd36-4e6c-bd28-0f387fb8b381 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:14.693205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2886e-bd36-4e6c-bd28-0f387fb8b381 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:14.724894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:14.729985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6051ef-5065-44cf-97fc-4b708a9f6c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:14.730668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6051ef-5065-44cf-97fc-4b708a9f6c3e None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:14.730957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6051ef-5065-44cf-97fc-4b708a9f6c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:14.731239 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6051ef-5065-44cf-97fc-4b708a9f6c3e None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:14.732000 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6051ef-5065-44cf-97fc-4b708a9f6c3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:14.732311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6051ef-5065-44cf-97fc-4b708a9f6c3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:14.733724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6051ef-5065-44cf-97fc-4b708a9f6c3e None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:14.743682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 634/1250] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:14.749692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049d496-09df-4c75-a204-0bda9e105cd6 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:14.831047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:14.854324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049d496-09df-4c75-a204-0bda9e105cd6 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:14.888619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dbfb7-d439-4c81-9310-91a2ee1131b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:14.889541 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dbfb7-d439-4c81-9310-91a2ee1131b0 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:14.889824 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dbfb7-d439-4c81-9310-91a2ee1131b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:14.890102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dbfb7-d439-4c81-9310-91a2ee1131b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:14.901757 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dbfb7-d439-4c81-9310-91a2ee1131b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:14.903840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dbfb7-d439-4c81-9310-91a2ee1131b0 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:14.921594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-bc8dbfb7-d439-4c81-9310-91a2ee1131b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:14.921594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:14.921594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:14.921594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:14.921594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:14.921594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:14.921594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:14.921594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:14.921594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:14.921594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:14.921594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:14.921594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:14.921594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:06:14.921594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:06:14.921594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:14.923412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:14.924555 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:06:14.924555 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:06:14.924555 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:14.924555 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:06:14.924555 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 617/1251] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:14.929080 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6051ef-5065-44cf-97fc-4b708a9f6c3e None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:15.029971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6051ef-5065-44cf-97fc-4b708a9f6c3e None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:15.066122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff824a8a-2c56-4af4-90e4-54fa0576f6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:15.066787 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff824a8a-2c56-4af4-90e4-54fa0576f6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:15.067069 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff824a8a-2c56-4af4-90e4-54fa0576f6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:15.067344 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff824a8a-2c56-4af4-90e4-54fa0576f6b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:15.068073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff824a8a-2c56-4af4-90e4-54fa0576f6b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:15.068374 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff824a8a-2c56-4af4-90e4-54fa0576f6b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:15.069794 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff824a8a-2c56-4af4-90e4-54fa0576f6b1 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:15.082661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 635/1252] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:15.088617 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dbfb7-d439-4c81-9310-91a2ee1131b0 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:15.190398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dbfb7-d439-4c81-9310-91a2ee1131b0 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:15.216720 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:15.224907 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ff755-43d0-4316-ab2d-ce68b59af91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:15.225741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ff755-43d0-4316-ab2d-ce68b59af91f None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:15.226023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ff755-43d0-4316-ab2d-ce68b59af91f None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:15.226327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ff755-43d0-4316-ab2d-ce68b59af91f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:15.233870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ff755-43d0-4316-ab2d-ce68b59af91f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:15.235294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ff755-43d0-4316-ab2d-ce68b59af91f None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-1d5ff755-43d0-4316-ab2d-ce68b59af91f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:06:15.249372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:15.250506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:15.252241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:15.252241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:15.252241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:06:15.252241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:06:15.252241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:06:15.252241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:06:15.252241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:06:15.252241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:06:15.252241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:06:15.252241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:06:15.252241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 618/1253] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:15.257434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff824a8a-2c56-4af4-90e4-54fa0576f6b1 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:15.312319 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:15.355549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff824a8a-2c56-4af4-90e4-54fa0576f6b1 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:15.390174 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d372b0-05ca-4e61-8f2d-184766c314d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:15.390806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d372b0-05ca-4e61-8f2d-184766c314d1 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:15.391092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d372b0-05ca-4e61-8f2d-184766c314d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:15.391370 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d372b0-05ca-4e61-8f2d-184766c314d1 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:15.392835 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d372b0-05ca-4e61-8f2d-184766c314d1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:15.393174 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d372b0-05ca-4e61-8f2d-184766c314d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:15.394616 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d372b0-05ca-4e61-8f2d-184766c314d1 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:15.412051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 636/1254] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Jun 15 10:06:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:15.418252 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ff755-43d0-4316-ab2d-ce68b59af91f None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:15.522303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ff755-43d0-4316-ab2d-ce68b59af91f None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:15.557602 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41b470-ba6b-4c7f-b060-3166e8fd7daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:15.558259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41b470-ba6b-4c7f-b060-3166e8fd7daf None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:15.558546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41b470-ba6b-4c7f-b060-3166e8fd7daf None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:15.558827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41b470-ba6b-4c7f-b060-3166e8fd7daf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:15.613785 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:15.628040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41b470-ba6b-4c7f-b060-3166e8fd7daf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:15.629556 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41b470-ba6b-4c7f-b060-3166e8fd7daf None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:15.728379 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 619/1255] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:06:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 15 10:06:15.734304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d372b0-05ca-4e61-8f2d-184766c314d1 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:15.836600 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d372b0-05ca-4e61-8f2d-184766c314d1 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:15.850391 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:15.871442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d12de7-79e8-4b5b-9a99-3469b1604e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:15.872165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d12de7-79e8-4b5b-9a99-3469b1604e61 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:15.872449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d12de7-79e8-4b5b-9a99-3469b1604e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:15.872724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d12de7-79e8-4b5b-9a99-3469b1604e61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:15.883838 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d12de7-79e8-4b5b-9a99-3469b1604e61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:15.885349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d12de7-79e8-4b5b-9a99-3469b1604e61 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:15.902910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-76d12de7-79e8-4b5b-9a99-3469b1604e61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:15.902910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:15.902910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:15.902910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:15.902910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:15.902910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:15.902910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:15.902910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:15.902910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:15.902910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:15.902910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:15.902910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:15.902910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:06:15.902910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:06:15.902910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:15.903984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:15.905063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:06:15.905063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:06:15.905063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:15.905063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:15.905063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 637/1256] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:06:15.910648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41b470-ba6b-4c7f-b060-3166e8fd7daf None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:15.998581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:16.011482 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41b470-ba6b-4c7f-b060-3166e8fd7daf None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:16.046103 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f945cd-4b4a-409d-9832-1d72d7cc0a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:16.046768 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f945cd-4b4a-409d-9832-1d72d7cc0a79 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:16.047046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f945cd-4b4a-409d-9832-1d72d7cc0a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:16.047321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f945cd-4b4a-409d-9832-1d72d7cc0a79 None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:16.048096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f945cd-4b4a-409d-9832-1d72d7cc0a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:16.048994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f945cd-4b4a-409d-9832-1d72d7cc0a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:16.050979 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f945cd-4b4a-409d-9832-1d72d7cc0a79 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:16.064790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 620/1257] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:16.071206 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d12de7-79e8-4b5b-9a99-3469b1604e61 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:16.184875 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d12de7-79e8-4b5b-9a99-3469b1604e61 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:16.220074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b5db5-c528-450e-818f-37d9832bb6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:16.220871 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b5db5-c528-450e-818f-37d9832bb6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:16.225078 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b5db5-c528-450e-818f-37d9832bb6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:16.225078 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b5db5-c528-450e-818f-37d9832bb6c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:16.228031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b5db5-c528-450e-818f-37d9832bb6c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:16.229519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b5db5-c528-450e-818f-37d9832bb6c6 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:16.243721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-693b5db5-c528-450e-818f-37d9832bb6c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:06:16.243721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:16.243721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:16.243721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:16.243721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:16.243721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:16.243721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:16.243721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:16.243721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:16.243721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:16.243721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:16.243721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:16.243721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:06:16.243721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:06:16.243721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:16.244692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:16.245611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:06:16.245611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:06:16.245611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:06:16.245611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:06:16.245611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:06:16.245611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:06:16.245611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:06:16.245611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:16.245611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 638/1258] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:06:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:16.251062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f945cd-4b4a-409d-9832-1d72d7cc0a79 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:16.351419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f945cd-4b4a-409d-9832-1d72d7cc0a79 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:16.385890 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca2d31-f08a-4e13-b3c2-f5ca6fc0c02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:16.386552 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca2d31-f08a-4e13-b3c2-f5ca6fc0c02a None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:16.386835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca2d31-f08a-4e13-b3c2-f5ca6fc0c02a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:16.387109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca2d31-f08a-4e13-b3c2-f5ca6fc0c02a None admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:16.387901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca2d31-f08a-4e13-b3c2-f5ca6fc0c02a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:16.388214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca2d31-f08a-4e13-b3c2-f5ca6fc0c02a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:16.389701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca2d31-f08a-4e13-b3c2-f5ca6fc0c02a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:16.402466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 621/1259] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:06:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:16.409438 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b5db5-c528-450e-818f-37d9832bb6c6 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:16.435736 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:16.513032 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b5db5-c528-450e-818f-37d9832bb6c6 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:16.548716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32fad7-5fbb-49e5-9166-9bc9bfc9bbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:16.549411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32fad7-5fbb-49e5-9166-9bc9bfc9bbe2 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:16.549698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32fad7-5fbb-49e5-9166-9bc9bfc9bbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:16.550231 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32fad7-5fbb-49e5-9166-9bc9bfc9bbe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:16.551068 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32fad7-5fbb-49e5-9166-9bc9bfc9bbe2 None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=12899a7a59b54e3199b277ff94e30d42) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:16.551382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32fad7-5fbb-49e5-9166-9bc9bfc9bbe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:16.552819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32fad7-5fbb-49e5-9166-9bc9bfc9bbe2 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:16.558328 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 639/1260] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:06:16 2022] GET /identity/v3/role_assignments?user.id=12899a7a59b54e3199b277ff94e30d42&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:06:18.696321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72160788-6083-4d1c-98d1-8439d9afdf63 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:18.696903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72160788-6083-4d1c-98d1-8439d9afdf63 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:18.696903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72160788-6083-4d1c-98d1-8439d9afdf63 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:18.698309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 622/1261] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:06:18.704535 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc9acd-d30d-4b96-ae9e-c29daf3ac05e None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:18.705233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc9acd-d30d-4b96-ae9e-c29daf3ac05e None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:18.705899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc9acd-d30d-4b96-ae9e-c29daf3ac05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:18.723863 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-00dc9acd-d30d-4b96-ae9e-c29daf3ac05e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:06:18.725190 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-00dc9acd-d30d-4b96-ae9e-c29daf3ac05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:06:18.794831 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 640/1262] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:06:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 15 10:06:18.819451 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72160788-6083-4d1c-98d1-8439d9afdf63 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:18.830654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:18.921338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72160788-6083-4d1c-98d1-8439d9afdf63 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:18.955575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fca0ae-f5bd-45d5-9725-4cc046865550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:18.956374 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fca0ae-f5bd-45d5-9725-4cc046865550 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:18.956660 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fca0ae-f5bd-45d5-9725-4cc046865550 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:18.956938 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fca0ae-f5bd-45d5-9725-4cc046865550 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:18.966977 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fca0ae-f5bd-45d5-9725-4cc046865550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-f3fca0ae-f5bd-45d5-9725-4cc046865550 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:06:18.975695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:18.976799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:18.986954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:18.986954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:18.986954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:18.986954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:18.986954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:06:18.986954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:06:18.986954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:06:18.986954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:06:18.986954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:06:18.986954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:06:18.986954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:06:18.986954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:06:18.986954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 623/1263] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:06:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:18.987868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc9acd-d30d-4b96-ae9e-c29daf3ac05e None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:19.002671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:19.081980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc9acd-d30d-4b96-ae9e-c29daf3ac05e None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:19.117510 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ed56f-746f-481f-a1f5-56c7769aa4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:19.118181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ed56f-746f-481f-a1f5-56c7769aa4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:19.118462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ed56f-746f-481f-a1f5-56c7769aa4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:19.118737 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ed56f-746f-481f-a1f5-56c7769aa4f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:19.119502 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ed56f-746f-481f-a1f5-56c7769aa4f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:19.119791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ed56f-746f-481f-a1f5-56c7769aa4f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:19.121188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ed56f-746f-481f-a1f5-56c7769aa4f5 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:19.129828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 641/1264] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:06:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:06:19.135683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fca0ae-f5bd-45d5-9725-4cc046865550 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:19.203074 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:19.235855 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fca0ae-f5bd-45d5-9725-4cc046865550 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:19.268822 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf01a21-c7dc-43e5-ab5d-ad21a99e381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:19.269652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf01a21-c7dc-43e5-ab5d-ad21a99e381d None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:19.269936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf01a21-c7dc-43e5-ab5d-ad21a99e381d None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:19.270213 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf01a21-c7dc-43e5-ab5d-ad21a99e381d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:19.280445 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf01a21-c7dc-43e5-ab5d-ad21a99e381d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:19.281904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf01a21-c7dc-43e5-ab5d-ad21a99e381d None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:19.299344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-faf01a21-c7dc-43e5-ab5d-ad21a99e381d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:19.299344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:19.299344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:19.299344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:19.299344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:19.299344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:19.299344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:19.299344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:19.299344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:19.299344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:19.299344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:19.299344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:19.299344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:06:19.299344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:06:19.299344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:19.300487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:19.301799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:06:19.301799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:06:19.301799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:19.301799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:06:19.301799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 624/1265] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:19.306696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ed56f-746f-481f-a1f5-56c7769aa4f5 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:19.390266 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:19.410588 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ed56f-746f-481f-a1f5-56c7769aa4f5 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:19.443905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674b4cd-3da0-4246-af9a-3077c2f246e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:19.444572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674b4cd-3da0-4246-af9a-3077c2f246e9 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:19.444860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674b4cd-3da0-4246-af9a-3077c2f246e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:19.445172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674b4cd-3da0-4246-af9a-3077c2f246e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:19.445899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674b4cd-3da0-4246-af9a-3077c2f246e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:19.446378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674b4cd-3da0-4246-af9a-3077c2f246e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:19.447643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1674b4cd-3da0-4246-af9a-3077c2f246e9 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:19.460952 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 642/1266] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:19.466736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf01a21-c7dc-43e5-ab5d-ad21a99e381d None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:19.571358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf01a21-c7dc-43e5-ab5d-ad21a99e381d None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:19.592056 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:19.606719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfaed1f-4255-4f1f-83d6-2717ab97df49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:19.607527 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfaed1f-4255-4f1f-83d6-2717ab97df49 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:19.607813 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfaed1f-4255-4f1f-83d6-2717ab97df49 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:19.608092 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfaed1f-4255-4f1f-83d6-2717ab97df49 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:19.617108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfaed1f-4255-4f1f-83d6-2717ab97df49 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:19.618850 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfaed1f-4255-4f1f-83d6-2717ab97df49 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-3bfaed1f-4255-4f1f-83d6-2717ab97df49 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:06:19.634861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:19.635843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:19.636829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:19.636829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:19.636829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:06:19.636829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:06:19.636829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:06:19.636829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:06:19.636829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:06:19.636829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:06:19.636829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:06:19.636829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:06:19.636829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 625/1267] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:19.643077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674b4cd-3da0-4246-af9a-3077c2f246e9 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:19.742792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674b4cd-3da0-4246-af9a-3077c2f246e9 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:19.777655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb362c-779b-4910-926a-aaa527dc3e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:19.778321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb362c-779b-4910-926a-aaa527dc3e65 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:19.778602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb362c-779b-4910-926a-aaa527dc3e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:19.778877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb362c-779b-4910-926a-aaa527dc3e65 None admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:19.779822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb362c-779b-4910-926a-aaa527dc3e65 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:19.780131 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb362c-779b-4910-926a-aaa527dc3e65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:19.781589 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb362c-779b-4910-926a-aaa527dc3e65 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:19.790959 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:19.801601 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 643/1268] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Jun 15 10:06:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:19.807554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfaed1f-4255-4f1f-83d6-2717ab97df49 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:19.913593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfaed1f-4255-4f1f-83d6-2717ab97df49 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:19.947752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5d6bc-b4ca-44f8-8f08-f1ff5662e8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:19.948546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5d6bc-b4ca-44f8-8f08-f1ff5662e8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:19.948829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5d6bc-b4ca-44f8-8f08-f1ff5662e8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:19.949133 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5d6bc-b4ca-44f8-8f08-f1ff5662e8c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:19.960224 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5d6bc-b4ca-44f8-8f08-f1ff5662e8c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:19.961689 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5d6bc-b4ca-44f8-8f08-f1ff5662e8c7 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:19.979761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-9af5d6bc-b4ca-44f8-8f08-f1ff5662e8c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:19.979761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:19.979761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:19.979761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:19.979761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:19.979761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:19.979761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:19.979761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:19.979761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:19.979761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:19.979761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:19.979761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:19.979761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:06:19.979761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:06:19.979761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:19.980821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:19.983675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:06:19.983675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:06:19.983675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:19.983675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:06:19.983675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 626/1269] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:19.987394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb362c-779b-4910-926a-aaa527dc3e65 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:20.085915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb362c-779b-4910-926a-aaa527dc3e65 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:20.119429 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850be00-55e1-4f1b-b7dd-f670ac7123f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:20.120092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850be00-55e1-4f1b-b7dd-f670ac7123f0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:20.120375 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850be00-55e1-4f1b-b7dd-f670ac7123f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:20.120653 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850be00-55e1-4f1b-b7dd-f670ac7123f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:20.121409 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850be00-55e1-4f1b-b7dd-f670ac7123f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:20.121716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850be00-55e1-4f1b-b7dd-f670ac7123f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:20.123091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850be00-55e1-4f1b-b7dd-f670ac7123f0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:20.134196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 644/1270] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:20.139941 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5d6bc-b4ca-44f8-8f08-f1ff5662e8c7 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:20.172715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:20.240908 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5d6bc-b4ca-44f8-8f08-f1ff5662e8c7 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:20.275665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ee910-bd08-477b-b3d9-d85a644a2262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:20.276552 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ee910-bd08-477b-b3d9-d85a644a2262 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:20.276831 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ee910-bd08-477b-b3d9-d85a644a2262 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:20.277133 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ee910-bd08-477b-b3d9-d85a644a2262 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:20.284833 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ee910-bd08-477b-b3d9-d85a644a2262 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:20.290042 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ee910-bd08-477b-b3d9-d85a644a2262 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:20.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-213ee910-bd08-477b-b3d9-d85a644a2262 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:06:20.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:20.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:20.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:20.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:20.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:20.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:20.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:20.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:20.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:20.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:20.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:20.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:06:20.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:06:20.301114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:20.302104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:20.303108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:06:20.303108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:06:20.303108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:06:20.303108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:06:20.303108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:06:20.303108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:06:20.303108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:06:20.303108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:06:20.303108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 627/1271] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:06:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:20.308517 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850be00-55e1-4f1b-b7dd-f670ac7123f0 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:20.370997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:20.412356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850be00-55e1-4f1b-b7dd-f670ac7123f0 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:20.447673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e301b7-bcf8-46c8-bfd7-6c4547b62ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:20.448239 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e301b7-bcf8-46c8-bfd7-6c4547b62ddc None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:20.448523 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e301b7-bcf8-46c8-bfd7-6c4547b62ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:20.448801 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e301b7-bcf8-46c8-bfd7-6c4547b62ddc None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:20.449632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e301b7-bcf8-46c8-bfd7-6c4547b62ddc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:20.449944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e301b7-bcf8-46c8-bfd7-6c4547b62ddc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:20.451379 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e301b7-bcf8-46c8-bfd7-6c4547b62ddc None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:20.464154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 645/1272] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:06:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:20.470386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ee910-bd08-477b-b3d9-d85a644a2262 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:20.567055 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:20.575898 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ee910-bd08-477b-b3d9-d85a644a2262 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:20.610079 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382eaf2c-fa5e-4aa5-a62f-b882a9699a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:20.611015 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382eaf2c-fa5e-4aa5-a62f-b882a9699a9a None admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:20.611015 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382eaf2c-fa5e-4aa5-a62f-b882a9699a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:20.611615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382eaf2c-fa5e-4aa5-a62f-b882a9699a9a None admin] PATH_INFO: `/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/12899a7a59b54e3199b277ff94e30d42/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:20.613527 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382eaf2c-fa5e-4aa5-a62f-b882a9699a9a None admin] RBAC: Authorizing `identity:create_grant(project_id=db47f1bb607c4d1a9c346fdc80ecadfa, user_id=12899a7a59b54e3199b277ff94e30d42, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:20.614955 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382eaf2c-fa5e-4aa5-a62f-b882a9699a9a None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:20.714280 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 628/1273] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jun 15 10:06:20 2022] PUT /identity/v3/projects/db47f1bb607c4d1a9c346fdc80ecadfa/users/12899a7a59b54e3199b277ff94e30d42/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:06:22.904024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d91981-d976-4c42-bc43-aa17294aefa7 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:22.904576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d91981-d976-4c42-bc43-aa17294aefa7 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:22.904576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d91981-d976-4c42-bc43-aa17294aefa7 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:22.905935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 646/1274] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:06:22.911543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fea54-aad7-4ebd-86e8-2777ec01aae7 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:22.911997 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fea54-aad7-4ebd-86e8-2777ec01aae7 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:22.912096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fea54-aad7-4ebd-86e8-2777ec01aae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:22.928967 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-8f6fea54-aad7-4ebd-86e8-2777ec01aae7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:06:22.930188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-8f6fea54-aad7-4ebd-86e8-2777ec01aae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:06:22.997960 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 629/1275] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:06:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:06:23.022124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d91981-d976-4c42-bc43-aa17294aefa7 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:23.125934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d91981-d976-4c42-bc43-aa17294aefa7 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:23.144345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:23.161924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f194da2-8031-4fe9-80fe-4b33bb5365df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:23.162632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f194da2-8031-4fe9-80fe-4b33bb5365df None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:23.162912 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f194da2-8031-4fe9-80fe-4b33bb5365df None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:23.163185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f194da2-8031-4fe9-80fe-4b33bb5365df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:23.173225 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f194da2-8031-4fe9-80fe-4b33bb5365df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-3f194da2-8031-4fe9-80fe-4b33bb5365df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 15 10:06:23.181443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:23.183390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:23.184385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:23.184385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:23.184385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:23.184385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:23.184385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 15 10:06:23.184385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 15 10:06:23.184385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 15 10:06:23.184385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 15 10:06:23.184385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 15 10:06:23.184385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 15 10:06:23.184385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 15 10:06:23.184385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:23.184385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 647/1276] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jun 15 10:06:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:23.188896 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fea54-aad7-4ebd-86e8-2777ec01aae7 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:23.272311 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:23.290489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fea54-aad7-4ebd-86e8-2777ec01aae7 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:23.323664 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7524bab-d4c0-4002-9def-790d57fe7672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:23.324331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7524bab-d4c0-4002-9def-790d57fe7672 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:23.324615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7524bab-d4c0-4002-9def-790d57fe7672 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:23.324897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7524bab-d4c0-4002-9def-790d57fe7672 None admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:23.325681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7524bab-d4c0-4002-9def-790d57fe7672 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:23.325993 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7524bab-d4c0-4002-9def-790d57fe7672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:23.327354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7524bab-d4c0-4002-9def-790d57fe7672 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:23.335975 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 630/1277] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jun 15 10:06:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:23.341839 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f194da2-8031-4fe9-80fe-4b33bb5365df None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:23.443718 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f194da2-8031-4fe9-80fe-4b33bb5365df None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:23.477397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c2c63-90ae-44f5-8275-26bace3a53ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:23.478187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c2c63-90ae-44f5-8275-26bace3a53ba None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:23.478469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c2c63-90ae-44f5-8275-26bace3a53ba None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:23.478743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030c2c63-90ae-44f5-8275-26bace3a53ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:23.489338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c2c63-90ae-44f5-8275-26bace3a53ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:23.493776 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030c2c63-90ae-44f5-8275-26bace3a53ba None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:23.508431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-030c2c63-90ae-44f5-8275-26bace3a53ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:23.508431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:23.508431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:23.508431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:23.508431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:23.508431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:23.508431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:23.508431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:23.508431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:23.508431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:23.508431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:23.508431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:23.508431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:06:23.508431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:06:23.508431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:23.509561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:23.510489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:06:23.510489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:06:23.510489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:23.510489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:23.510489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 648/1278] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:23.515512 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7524bab-d4c0-4002-9def-790d57fe7672 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:23.621635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7524bab-d4c0-4002-9def-790d57fe7672 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:23.656383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdfcb50-befd-48f2-8ea4-a50a74f8c6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:23.657080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfcb50-befd-48f2-8ea4-a50a74f8c6dc None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:23.657362 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfcb50-befd-48f2-8ea4-a50a74f8c6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:23.657632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfcb50-befd-48f2-8ea4-a50a74f8c6dc None admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:23.658493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdfcb50-befd-48f2-8ea4-a50a74f8c6dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:23.658710 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdfcb50-befd-48f2-8ea4-a50a74f8c6dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:23.660052 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdfcb50-befd-48f2-8ea4-a50a74f8c6dc None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:23.676006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 631/1279] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:23.682053 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c2c63-90ae-44f5-8275-26bace3a53ba None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:23.714446 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:23.790401 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030c2c63-90ae-44f5-8275-26bace3a53ba None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:23.834488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3be4bb-8d67-4e49-9196-644bbd6c4b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:23.835315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3be4bb-8d67-4e49-9196-644bbd6c4b67 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:23.835596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3be4bb-8d67-4e49-9196-644bbd6c4b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:23.835892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3be4bb-8d67-4e49-9196-644bbd6c4b67 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:23.844719 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3be4bb-8d67-4e49-9196-644bbd6c4b67 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:23.847425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3be4bb-8d67-4e49-9196-644bbd6c4b67 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-6f3be4bb-8d67-4e49-9196-644bbd6c4b67 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 15 10:06:23.863518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:23.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:23.866565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:23.866565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:23.866565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 15 10:06:23.866565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 15 10:06:23.866565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 15 10:06:23.866565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 15 10:06:23.866565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 15 10:06:23.866565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 15 10:06:23.866565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 15 10:06:23.866565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:23.866565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 649/1280] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:23.873268 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdfcb50-befd-48f2-8ea4-a50a74f8c6dc None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:23.877326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:23.971667 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdfcb50-befd-48f2-8ea4-a50a74f8c6dc None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:24.006938 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa7c2e-8c2a-44ee-a767-aef27121d2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:24.007604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa7c2e-8c2a-44ee-a767-aef27121d2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:24.007884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa7c2e-8c2a-44ee-a767-aef27121d2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:24.008153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aa7c2e-8c2a-44ee-a767-aef27121d2d7 None admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:24.009011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa7c2e-8c2a-44ee-a767-aef27121d2d7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:24.009319 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa7c2e-8c2a-44ee-a767-aef27121d2d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:24.010781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aa7c2e-8c2a-44ee-a767-aef27121d2d7 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:24.028928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 632/1281] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Jun 15 10:06:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:24.035554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3be4bb-8d67-4e49-9196-644bbd6c4b67 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:24.128845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:24.138232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3be4bb-8d67-4e49-9196-644bbd6c4b67 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:24.172489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b6db7-3881-442b-800d-8e77b337fdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:24.173166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b6db7-3881-442b-800d-8e77b337fdba None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:24.173453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b6db7-3881-442b-800d-8e77b337fdba None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:24.173966 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b6db7-3881-442b-800d-8e77b337fdba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:24.243202 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b6db7-3881-442b-800d-8e77b337fdba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:24.244571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b6db7-3881-442b-800d-8e77b337fdba None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:24.333167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:24.342803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 650/1282] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jun 15 10:06:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 15 10:06:24.349169 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa7c2e-8c2a-44ee-a767-aef27121d2d7 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:24.410341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:24.449319 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aa7c2e-8c2a-44ee-a767-aef27121d2d7 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:24.482886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb1765-731c-41a8-9a9e-3ba6e3e500b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:24.483671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb1765-731c-41a8-9a9e-3ba6e3e500b5 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:24.483943 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb1765-731c-41a8-9a9e-3ba6e3e500b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:24.484211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb1765-731c-41a8-9a9e-3ba6e3e500b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:24.494736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb1765-731c-41a8-9a9e-3ba6e3e500b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:24.496165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb1765-731c-41a8-9a9e-3ba6e3e500b5 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:24.514144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-edfb1765-731c-41a8-9a9e-3ba6e3e500b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:24.514144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:24.514144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:24.514144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:24.514144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:24.514144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:24.514144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:24.514144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:24.514144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:24.514144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:24.514144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:24.514144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:24.514144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 15 10:06:24.514144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 15 10:06:24.514144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:24.515229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:24.516292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 15 10:06:24.516292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 15 10:06:24.516292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 15 10:06:24.516292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:06:24.516292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 633/1283] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:24.521144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b6db7-3881-442b-800d-8e77b337fdba None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:24.624818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b6db7-3881-442b-800d-8e77b337fdba None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:24.660140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4aea95-0189-4766-b831-a9e38201db65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:24.660805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aea95-0189-4766-b831-a9e38201db65 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:24.661126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aea95-0189-4766-b831-a9e38201db65 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:24.661435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aea95-0189-4766-b831-a9e38201db65 None admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:24.662131 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4aea95-0189-4766-b831-a9e38201db65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:24.662440 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4aea95-0189-4766-b831-a9e38201db65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:24.663811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4aea95-0189-4766-b831-a9e38201db65 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:24.675808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 651/1284] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jun 15 10:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:24.681552 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb1765-731c-41a8-9a9e-3ba6e3e500b5 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:24.783801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb1765-731c-41a8-9a9e-3ba6e3e500b5 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:24.797513 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:24.819360 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede0844-3ec5-411a-a0cf-d3db587a714e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:24.820156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede0844-3ec5-411a-a0cf-d3db587a714e None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:24.820443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede0844-3ec5-411a-a0cf-d3db587a714e None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:24.820719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede0844-3ec5-411a-a0cf-d3db587a714e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:24.827838 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede0844-3ec5-411a-a0cf-d3db587a714e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:24.833936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede0844-3ec5-411a-a0cf-d3db587a714e None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:24.843802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application [None req-aede0844-3ec5-411a-a0cf-d3db587a714e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:06:24.843802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:24.843802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:24.843802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:24.843802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:24.843802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:24.843802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:24.843802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:24.843802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:24.843802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:24.843802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:24.843802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:24.843802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 15 10:06:24.843802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 15 10:06:24.843802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:24.844844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:24.845918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:06:24.845918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:06:24.845918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:06:24.845918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:06:24.845918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:06:24.845918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:06:24.845918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 15 10:06:24.845918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: ERROR keystone.server.flask.application Jun 15 10:06:24.845918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 634/1285] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jun 15 10:06:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:24.851096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4aea95-0189-4766-b831-a9e38201db65 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:24.903373 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:24.953581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4aea95-0189-4766-b831-a9e38201db65 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:24.987515 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02f0f4-71a3-42a8-9461-68ff877658d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:24.988187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02f0f4-71a3-42a8-9461-68ff877658d0 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:24.988462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02f0f4-71a3-42a8-9461-68ff877658d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:24.988744 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02f0f4-71a3-42a8-9461-68ff877658d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:24.989578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02f0f4-71a3-42a8-9461-68ff877658d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:24.989893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02f0f4-71a3-42a8-9461-68ff877658d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:24.991368 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02f0f4-71a3-42a8-9461-68ff877658d0 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:25.003132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 652/1286] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jun 15 10:06:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:25.009211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede0844-3ec5-411a-a0cf-d3db587a714e None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:25.111034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede0844-3ec5-411a-a0cf-d3db587a714e None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:25.145409 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac450ead-1753-4a9c-b10c-988b779dcbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:25.146054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac450ead-1753-4a9c-b10c-988b779dcbb1 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:25.146338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac450ead-1753-4a9c-b10c-988b779dcbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:25.146609 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac450ead-1753-4a9c-b10c-988b779dcbb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:25.147484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac450ead-1753-4a9c-b10c-988b779dcbb1 None admin] RBAC: Adding query filter params (role.id=3dbbfbc1ca2348bc87483338420ce4a8, scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa, user.id=12899a7a59b54e3199b277ff94e30d42) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:25.147801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac450ead-1753-4a9c-b10c-988b779dcbb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:25.149248 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac450ead-1753-4a9c-b10c-988b779dcbb1 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:25.154983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 635/1287] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jun 15 10:06:25 2022] GET /identity/v3/role_assignments?user.id=12899a7a59b54e3199b277ff94e30d42&scope.project.id=db47f1bb607c4d1a9c346fdc80ecadfa&role.id=3dbbfbc1ca2348bc87483338420ce4a8 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:27.360112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070b17d-5fb9-4f24-9219-fa62656673fd None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:27.360939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070b17d-5fb9-4f24-9219-fa62656673fd None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:27.361295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070b17d-5fb9-4f24-9219-fa62656673fd None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:27.363170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 653/1288] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:06:27.368705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2909d9f-f4af-48fb-a69d-4a5883d9932d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:27.369182 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2909d9f-f4af-48fb-a69d-4a5883d9932d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:27.376032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2909d9f-f4af-48fb-a69d-4a5883d9932d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:27.389791 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-e2909d9f-f4af-48fb-a69d-4a5883d9932d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:06:27.390990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e2909d9f-f4af-48fb-a69d-4a5883d9932d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:06:27.428504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:27.473110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 636/1289] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:06:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:06:27.493260 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070b17d-5fb9-4f24-9219-fa62656673fd None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:27.608172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070b17d-5fb9-4f24-9219-fa62656673fd None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:27.625439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:27.645108 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d33342-aa63-4445-897a-a615ff5068b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:27.648238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d33342-aa63-4445-897a-a615ff5068b9 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:27.648238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d33342-aa63-4445-897a-a615ff5068b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:27.648238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d33342-aa63-4445-897a-a615ff5068b9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:27.648238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d33342-aa63-4445-897a-a615ff5068b9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:27.648598 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d33342-aa63-4445-897a-a615ff5068b9 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-30d33342-aa63-4445-897a-a615ff5068b9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:27.661092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:27.662315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:27.663195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:27.663195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:27.663195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:27.663195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:27.663195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:27.663195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:27.663195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:27.663195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:27.663195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:27.663195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:27.663195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:06:27.663195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:06:27.663195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:06:27.663195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:27.663195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 654/1290] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jun 15 10:06:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:27.668657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2909d9f-f4af-48fb-a69d-4a5883d9932d None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:27.770694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2909d9f-f4af-48fb-a69d-4a5883d9932d None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:27.805601 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68009be4-ed15-4cdc-a595-e7d3033a08b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:27.806270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68009be4-ed15-4cdc-a595-e7d3033a08b2 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:27.806556 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68009be4-ed15-4cdc-a595-e7d3033a08b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:27.806835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68009be4-ed15-4cdc-a595-e7d3033a08b2 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:27.807638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68009be4-ed15-4cdc-a595-e7d3033a08b2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:27.807956 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68009be4-ed15-4cdc-a595-e7d3033a08b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:27.809357 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68009be4-ed15-4cdc-a595-e7d3033a08b2 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:27.814151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 637/1291] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Jun 15 10:06:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:27.821221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d33342-aa63-4445-897a-a615ff5068b9 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:27.928934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d33342-aa63-4445-897a-a615ff5068b9 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:27.962621 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c852647-7d09-4294-a068-1d632a5d6810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:27.963277 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c852647-7d09-4294-a068-1d632a5d6810 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:27.963560 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c852647-7d09-4294-a068-1d632a5d6810 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:27.963836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c852647-7d09-4294-a068-1d632a5d6810 None admin] PATH_INFO: `/v3/services` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:27.964609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c852647-7d09-4294-a068-1d632a5d6810 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:27.964892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c852647-7d09-4294-a068-1d632a5d6810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:27.966276 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c852647-7d09-4294-a068-1d632a5d6810 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:27.971925 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 655/1292] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Jun 15 10:06:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:30.146474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fde85-84bd-4203-8399-9813436afe8f None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:30.147004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fde85-84bd-4203-8399-9813436afe8f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:30.147004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fde85-84bd-4203-8399-9813436afe8f None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:30.148402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 638/1293] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:06:30.153710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d71d06-96a3-4fba-9481-8388452b7ebb None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:30.154135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d71d06-96a3-4fba-9481-8388452b7ebb None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:30.154269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d71d06-96a3-4fba-9481-8388452b7ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:30.172042 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-f5d71d06-96a3-4fba-9481-8388452b7ebb None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:06:30.175015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f5d71d06-96a3-4fba-9481-8388452b7ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:06:30.204894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:30.242859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 656/1294] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:06:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:06:30.266741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fde85-84bd-4203-8399-9813436afe8f None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:30.298040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:30.370746 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fde85-84bd-4203-8399-9813436afe8f None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:30.406728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76277982-4672-4219-89b3-713cf2b38823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:30.406728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76277982-4672-4219-89b3-713cf2b38823 None admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:30.406728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76277982-4672-4219-89b3-713cf2b38823 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:30.407214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76277982-4672-4219-89b3-713cf2b38823 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:30.407670 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76277982-4672-4219-89b3-713cf2b38823 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:30.409118 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76277982-4672-4219-89b3-713cf2b38823 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:30.473476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 639/1295] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jun 15 10:06:30 2022] POST /identity/v3/services => generated 246 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:06:32.642233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48565a-cb18-4af4-91d7-54d46db61220 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:32.642806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48565a-cb18-4af4-91d7-54d46db61220 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:32.642806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48565a-cb18-4af4-91d7-54d46db61220 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:32.644207 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 657/1296] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:06:32.649744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb54c19-ca3c-4293-8967-0ff5d1f2f2eb None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:32.652660 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb54c19-ca3c-4293-8967-0ff5d1f2f2eb None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:32.652660 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb54c19-ca3c-4293-8967-0ff5d1f2f2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:32.667700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-fbb54c19-ca3c-4293-8967-0ff5d1f2f2eb None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:06:32.668872 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-fbb54c19-ca3c-4293-8967-0ff5d1f2f2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:06:32.742842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 640/1297] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:06:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:06:32.766570 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48565a-cb18-4af4-91d7-54d46db61220 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:32.870628 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48565a-cb18-4af4-91d7-54d46db61220 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:32.906222 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33ac32-f2ad-4ecb-bf79-b728c122ba68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:32.907022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33ac32-f2ad-4ecb-bf79-b728c122ba68 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:32.907309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33ac32-f2ad-4ecb-bf79-b728c122ba68 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:32.907590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33ac32-f2ad-4ecb-bf79-b728c122ba68 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:32.908251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33ac32-f2ad-4ecb-bf79-b728c122ba68 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:32.910317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33ac32-f2ad-4ecb-bf79-b728c122ba68 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-8b33ac32-f2ad-4ecb-bf79-b728c122ba68 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:32.922712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:32.923806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:32.924670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:32.924670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:32.924670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:32.924670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:32.924670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:32.924670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:32.924670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:32.924670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:32.924670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:32.924670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:32.924670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:06:32.924670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:06:32.924670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:06:32.924670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:32.924670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 658/1298] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jun 15 10:06:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 15 10:06:32.930928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb54c19-ca3c-4293-8967-0ff5d1f2f2eb None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:32.950023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:33.034129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb54c19-ca3c-4293-8967-0ff5d1f2f2eb None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:33.072682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c97438-a023-4703-9302-110605227270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:33.073397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c97438-a023-4703-9302-110605227270 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:33.073681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c97438-a023-4703-9302-110605227270 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:33.073959 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c97438-a023-4703-9302-110605227270 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:33.074684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c97438-a023-4703-9302-110605227270 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:33.074992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c97438-a023-4703-9302-110605227270 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:33.076384 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c97438-a023-4703-9302-110605227270 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:33.082222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 641/1299] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Jun 15 10:06:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:33.088493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33ac32-f2ad-4ecb-bf79-b728c122ba68 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:33.120353 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:33.190662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33ac32-f2ad-4ecb-bf79-b728c122ba68 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:33.224321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b97b3-be19-4751-8b9e-742285e91578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:33.225001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b97b3-be19-4751-8b9e-742285e91578 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:33.225290 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b97b3-be19-4751-8b9e-742285e91578 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:33.225572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b97b3-be19-4751-8b9e-742285e91578 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:33.226367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b97b3-be19-4751-8b9e-742285e91578 None admin] RBAC: Adding query filter params (interface=public, service_id=fa148ceeebeb4869b33fc8a9af6a7d60, region_id=RegionOne) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:33.226683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b97b3-be19-4751-8b9e-742285e91578 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:33.228116 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b97b3-be19-4751-8b9e-742285e91578 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:33.233265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 659/1300] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jun 15 10:06:33 2022] GET /identity/v3/endpoints?service_id=fa148ceeebeb4869b33fc8a9af6a7d60&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:33.239123 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c97438-a023-4703-9302-110605227270 None admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:33.321442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:33.338755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c97438-a023-4703-9302-110605227270 None admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:33.373176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee58f27-2458-4149-8c7d-5c8979753ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:33.373842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee58f27-2458-4149-8c7d-5c8979753ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:33.374126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee58f27-2458-4149-8c7d-5c8979753ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:33.374403 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee58f27-2458-4149-8c7d-5c8979753ef3 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:33.375085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee58f27-2458-4149-8c7d-5c8979753ef3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:33.376485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee58f27-2458-4149-8c7d-5c8979753ef3 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:33.384753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 642/1301] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jun 15 10:06:33 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:06:35.606901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5104d4-73ba-4053-a61e-fafdf5a90e21 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:35.607464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5104d4-73ba-4053-a61e-fafdf5a90e21 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:35.607464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5104d4-73ba-4053-a61e-fafdf5a90e21 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:35.608923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 660/1302] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:06:35.616180 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e81b68-f032-419f-b0f5-677470a773c2 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:35.616791 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e81b68-f032-419f-b0f5-677470a773c2 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:35.619083 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e81b68-f032-419f-b0f5-677470a773c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:35.648384 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-16e81b68-f032-419f-b0f5-677470a773c2 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:06:35.650341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-16e81b68-f032-419f-b0f5-677470a773c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:06:35.750478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 643/1303] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:06:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 15 10:06:35.779602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5104d4-73ba-4053-a61e-fafdf5a90e21 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:35.920082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:35.931573 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5104d4-73ba-4053-a61e-fafdf5a90e21 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:35.992499 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d8150-0075-420a-a88f-48350da115f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:35.994025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d8150-0075-420a-a88f-48350da115f2 None admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:35.994511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d8150-0075-420a-a88f-48350da115f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:35.994988 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d8150-0075-420a-a88f-48350da115f2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:35.996192 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d8150-0075-420a-a88f-48350da115f2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:35.998893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d8150-0075-420a-a88f-48350da115f2 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-579d8150-0075-420a-a88f-48350da115f2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:06:36.022943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:06:36.024084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:06:36.024934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:06:36.024934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:06:36.024934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:06:36.024934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:06:36.024934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:06:36.024934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:06:36.024934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:06:36.024934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:06:36.024934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:06:36.024934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:06:36.024934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 15 10:06:36.024934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 15 10:06:36.024934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 15 10:06:36.024934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:06:36.026009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 661/1304] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jun 15 10:06:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 15 10:06:36.035098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e81b68-f032-419f-b0f5-677470a773c2 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:36.128826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:36.158701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e81b68-f032-419f-b0f5-677470a773c2 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:36.197505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa4379-3dd1-4149-9d0b-1d1fbb8588a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:36.198202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa4379-3dd1-4149-9d0b-1d1fbb8588a2 None admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:36.198822 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa4379-3dd1-4149-9d0b-1d1fbb8588a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:36.199241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa4379-3dd1-4149-9d0b-1d1fbb8588a2 None admin] PATH_INFO: `/v3/services` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:36.200186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa4379-3dd1-4149-9d0b-1d1fbb8588a2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:06:36.200633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa4379-3dd1-4149-9d0b-1d1fbb8588a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:36.202557 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa4379-3dd1-4149-9d0b-1d1fbb8588a2 None admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:36.209169 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 644/1305] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Jun 15 10:06:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:06:36.218815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d8150-0075-420a-a88f-48350da115f2 None admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:06:36.376716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d8150-0075-420a-a88f-48350da115f2 None admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:06:36.422305 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfcd02-673b-4384-8b90-0bff31877ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:06:36.423051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfcd02-673b-4384-8b90-0bff31877ca6 None admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:06:36.423379 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfcd02-673b-4384-8b90-0bff31877ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:06:36.426792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfcd02-673b-4384-8b90-0bff31877ca6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:06:36.426792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfcd02-673b-4384-8b90-0bff31877ca6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:06:36.427225 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfcd02-673b-4384-8b90-0bff31877ca6 None admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:06:36.536636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 662/1306] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jun 15 10:06:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:06:46.205215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=30 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:06:46.529463 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:14.478694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5432f2-dce6-4d49-b32c-1e7f555319f5 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:14.479245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5432f2-dce6-4d49-b32c-1e7f555319f5 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:14.479245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5432f2-dce6-4d49-b32c-1e7f555319f5 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:14.487527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03858f0c-3100-47e9-8e54-5019aa103b58 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:14.488089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03858f0c-3100-47e9-8e54-5019aa103b58 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:14.488089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03858f0c-3100-47e9-8e54-5019aa103b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:14.490225 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 645/1307] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:13:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:13:14.574385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-03858f0c-3100-47e9-8e54-5019aa103b58 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:13:14.576624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-03858f0c-3100-47e9-8e54-5019aa103b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:14.690597 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 663/1308] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:13:14.743929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c728d-c276-4f70-bbeb-fecc7a5cd690 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:14.744635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c728d-c276-4f70-bbeb-fecc7a5cd690 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:14.745498 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c728d-c276-4f70-bbeb-fecc7a5cd690 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:14.747630 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 646/1309] 10.222.0.60 () {54 vars in 964 bytes} [Wed Jun 15 10:13:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:13:14.753804 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e929928-2f77-45e2-9645-32e88570c943 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:14.754100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e929928-2f77-45e2-9645-32e88570c943 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:14.754388 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e929928-2f77-45e2-9645-32e88570c943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:14.839971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-2e929928-2f77-45e2-9645-32e88570c943 None None] MFA Rules not processed for user `ae926ce83d464c768baddd90de31aa40`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:13:14.841440 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2e929928-2f77-45e2-9645-32e88570c943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:14.873465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:14.997530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 664/1310] 10.222.0.60 () {58 vars in 1061 bytes} [Wed Jun 15 10:13:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:13:15.006297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c728d-c276-4f70-bbeb-fecc7a5cd690 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:13:15.007962 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c97c728d-c276-4f70-bbeb-fecc7a5cd690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:15.251283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c728d-c276-4f70-bbeb-fecc7a5cd690 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:13:15.344684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb072a-c80b-49b5-a5a7-13e9c3c5dde4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:13:15.345626 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb072a-c80b-49b5-a5a7-13e9c3c5dde4 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:15.346019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb072a-c80b-49b5-a5a7-13e9c3c5dde4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:15.346364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb072a-c80b-49b5-a5a7-13e9c3c5dde4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:15.350435 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-8cfb072a-c80b-49b5-a5a7-13e9c3c5dde4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:15.414548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:15.480412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb072a-c80b-49b5-a5a7-13e9c3c5dde4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:13:15.482453 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb072a-c80b-49b5-a5a7-13e9c3c5dde4 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:13:15.506481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 647/1311] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:13:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:13:18.031001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b5646-cefa-4d60-9b44-aa91a0fd812a None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:18.031568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b5646-cefa-4d60-9b44-aa91a0fd812a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:18.031568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b5646-cefa-4d60-9b44-aa91a0fd812a None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:18.032914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 665/1312] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:13:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:13:18.038046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d2ca8-86f8-41e7-bc17-e58e01d24362 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:18.038434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d2ca8-86f8-41e7-bc17-e58e01d24362 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:18.038723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d2ca8-86f8-41e7-bc17-e58e01d24362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:18.122200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-c91d2ca8-86f8-41e7-bc17-e58e01d24362 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:13:18.124654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c91d2ca8-86f8-41e7-bc17-e58e01d24362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:18.224429 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 648/1313] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:13:18.276004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e7caf-1a0d-46f4-909c-aca7d0be986f None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:18.276387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e7caf-1a0d-46f4-909c-aca7d0be986f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:18.276738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e7caf-1a0d-46f4-909c-aca7d0be986f None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:18.278259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 666/1314] 10.222.0.60 () {54 vars in 964 bytes} [Wed Jun 15 10:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:13:18.284377 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da86fd-6318-4dba-a8c0-9ebeb2d03fcb None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:18.284606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da86fd-6318-4dba-a8c0-9ebeb2d03fcb None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:18.284886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da86fd-6318-4dba-a8c0-9ebeb2d03fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:18.296561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:18.377106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-01da86fd-6318-4dba-a8c0-9ebeb2d03fcb None None] MFA Rules not processed for user `ae926ce83d464c768baddd90de31aa40`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:13:18.378333 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-01da86fd-6318-4dba-a8c0-9ebeb2d03fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:18.464317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 649/1315] 10.222.0.60 () {58 vars in 1061 bytes} [Wed Jun 15 10:13:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:13:18.474781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e7caf-1a0d-46f4-909c-aca7d0be986f None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:13:18.476344 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-bd5e7caf-1a0d-46f4-909c-aca7d0be986f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:18.589996 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e7caf-1a0d-46f4-909c-aca7d0be986f None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:13:18.661341 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:18.663943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08119384-22a6-44be-b378-83d94e36f587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:13:18.664587 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08119384-22a6-44be-b378-83d94e36f587 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:18.664867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08119384-22a6-44be-b378-83d94e36f587 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:18.665167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08119384-22a6-44be-b378-83d94e36f587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:18.666799 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-08119384-22a6-44be-b378-83d94e36f587 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:18.781758 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08119384-22a6-44be-b378-83d94e36f587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:13:18.783357 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08119384-22a6-44be-b378-83d94e36f587 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:13:18.802277 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 667/1316] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:13:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:13:21.756287 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d9779-53ed-4408-b63d-e884c4d6953b None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:21.756816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d9779-53ed-4408-b63d-e884c4d6953b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:21.756929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8d9779-53ed-4408-b63d-e884c4d6953b None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:21.758332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 650/1317] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:13:21.763966 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55a706-8d4d-4667-a5ef-08351c9b68ec None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:21.764360 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55a706-8d4d-4667-a5ef-08351c9b68ec None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:21.764360 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55a706-8d4d-4667-a5ef-08351c9b68ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:21.831174 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-fd55a706-8d4d-4667-a5ef-08351c9b68ec None None] MFA Rules not processed for user `cdcce87a6587441884e93573b6bd6d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:13:21.832474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-fd55a706-8d4d-4667-a5ef-08351c9b68ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:21.885657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:21.930276 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 668/1318] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:13:21 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:13:21.971332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d9779-53ed-4408-b63d-e884c4d6953b None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:13:21.972892 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3a8d9779-53ed-4408-b63d-e884c4d6953b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:22.076986 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8d9779-53ed-4408-b63d-e884c4d6953b None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:13:22.080559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:22.151108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8a1e2-78c3-4ebb-82e9-4a98ff3ba39c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:13:22.151679 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8a1e2-78c3-4ebb-82e9-4a98ff3ba39c service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:22.151958 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8a1e2-78c3-4ebb-82e9-4a98ff3ba39c service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:22.152310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8a1e2-78c3-4ebb-82e9-4a98ff3ba39c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:22.153904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a3b8a1e2-78c3-4ebb-82e9-4a98ff3ba39c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:22.267134 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8a1e2-78c3-4ebb-82e9-4a98ff3ba39c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:13:22.268565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b8a1e2-78c3-4ebb-82e9-4a98ff3ba39c service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:13:22.286614 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 651/1319] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:13:21 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:13:25.535935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81437b00-6ac3-4cdf-baf9-391fa6e3b7ba None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:25.536501 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81437b00-6ac3-4cdf-baf9-391fa6e3b7ba None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:25.536501 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81437b00-6ac3-4cdf-baf9-391fa6e3b7ba None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:25.537902 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 669/1320] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:13:25.543263 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70260d0b-5e99-4b90-b915-4a7543fba632 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:25.543711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70260d0b-5e99-4b90-b915-4a7543fba632 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:25.543806 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70260d0b-5e99-4b90-b915-4a7543fba632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:25.585561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:25.612897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-70260d0b-5e99-4b90-b915-4a7543fba632 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:13:25.614203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-70260d0b-5e99-4b90-b915-4a7543fba632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:25.714009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 652/1321] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:13:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:13:25.753666 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81437b00-6ac3-4cdf-baf9-391fa6e3b7ba None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:13:25.755193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-81437b00-6ac3-4cdf-baf9-391fa6e3b7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:25.863214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81437b00-6ac3-4cdf-baf9-391fa6e3b7ba None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:13:25.936008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375b386-616e-4a8f-9ed9-52d77e4c74b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:13:25.937284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375b386-616e-4a8f-9ed9-52d77e4c74b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:25.937624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375b386-616e-4a8f-9ed9-52d77e4c74b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:25.938073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375b386-616e-4a8f-9ed9-52d77e4c74b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:25.939715 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d375b386-616e-4a8f-9ed9-52d77e4c74b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:25.946783 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:26.046651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375b386-616e-4a8f-9ed9-52d77e4c74b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:13:26.048080 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375b386-616e-4a8f-9ed9-52d77e4c74b6 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:13:26.066456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 670/1322] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:13:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:13:29.324596 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdfcab-593d-4754-a5f3-d7cc1438a02a None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:29.325146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdfcab-593d-4754-a5f3-d7cc1438a02a None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:29.325224 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdfcab-593d-4754-a5f3-d7cc1438a02a None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:29.326736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 653/1323] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:13:29.332156 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c137e9-e2dd-4fec-8883-fe42f3b8a98f None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:29.332610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c137e9-e2dd-4fec-8883-fe42f3b8a98f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:29.332758 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c137e9-e2dd-4fec-8883-fe42f3b8a98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:29.406162 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-83c137e9-e2dd-4fec-8883-fe42f3b8a98f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:13:29.407431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-83c137e9-e2dd-4fec-8883-fe42f3b8a98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:29.472114 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:29.517511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 671/1324] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:13:29.552385 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdfcab-593d-4754-a5f3-d7cc1438a02a None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:13:29.554034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-cfcdfcab-593d-4754-a5f3-d7cc1438a02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:29.672138 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdfcab-593d-4754-a5f3-d7cc1438a02a None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:13:29.685129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:29.752675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1659a93-78f5-407a-ab58-5b83d38c2bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:13:29.753242 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1659a93-78f5-407a-ab58-5b83d38c2bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:29.753527 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1659a93-78f5-407a-ab58-5b83d38c2bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:29.753799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1659a93-78f5-407a-ab58-5b83d38c2bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:29.755463 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f1659a93-78f5-407a-ab58-5b83d38c2bd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:29.874022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1659a93-78f5-407a-ab58-5b83d38c2bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:13:29.875465 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1659a93-78f5-407a-ab58-5b83d38c2bd4 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:13:29.895089 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 654/1325] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:13:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:13:32.151765 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef4261-6248-4eb6-80ad-5f18b9652ba3 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:32.152327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef4261-6248-4eb6-80ad-5f18b9652ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:32.152327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef4261-6248-4eb6-80ad-5f18b9652ba3 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:32.153585 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 672/1326] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:13:32.159149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27571f-7a1b-419d-ac6b-69b3e3f7f223 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:32.159612 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27571f-7a1b-419d-ac6b-69b3e3f7f223 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:32.159713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b27571f-7a1b-419d-ac6b-69b3e3f7f223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:32.209847 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:32.228463 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-4b27571f-7a1b-419d-ac6b-69b3e3f7f223 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:13:32.229718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4b27571f-7a1b-419d-ac6b-69b3e3f7f223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:32.331125 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 655/1327] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:13:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:13:32.367913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef4261-6248-4eb6-80ad-5f18b9652ba3 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:13:32.369473 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c5ef4261-6248-4eb6-80ad-5f18b9652ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:32.476012 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef4261-6248-4eb6-80ad-5f18b9652ba3 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:13:32.556113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346f366-4235-4abc-a0ec-eb9bddf5eaed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:13:32.556782 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346f366-4235-4abc-a0ec-eb9bddf5eaed service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:32.557093 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346f366-4235-4abc-a0ec-eb9bddf5eaed service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:32.557367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346f366-4235-4abc-a0ec-eb9bddf5eaed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:32.559107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-8346f366-4235-4abc-a0ec-eb9bddf5eaed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:32.580033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:32.676400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346f366-4235-4abc-a0ec-eb9bddf5eaed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:13:32.677875 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346f366-4235-4abc-a0ec-eb9bddf5eaed service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:13:32.696398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 673/1328] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:13:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:13:34.957528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace6000-c9b2-447a-8014-d634a113cb09 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:34.958094 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace6000-c9b2-447a-8014-d634a113cb09 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:34.958094 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace6000-c9b2-447a-8014-d634a113cb09 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:34.959458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 656/1329] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:13:34.965077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070f762-ed53-40ac-bee4-042530b8eb8b None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:34.965559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070f762-ed53-40ac-bee4-042530b8eb8b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:34.965658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070f762-ed53-40ac-bee4-042530b8eb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:35.032417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-5070f762-ed53-40ac-bee4-042530b8eb8b None None] MFA Rules not processed for user `cdcce87a6587441884e93573b6bd6d5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:13:35.033658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5070f762-ed53-40ac-bee4-042530b8eb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:35.099155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:35.131846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 674/1330] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:13:34 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:13:35.169310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace6000-c9b2-447a-8014-d634a113cb09 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:13:35.171558 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-7ace6000-c9b2-447a-8014-d634a113cb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:35.282073 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace6000-c9b2-447a-8014-d634a113cb09 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:13:35.304267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:35.372298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84053c7d-01f9-45b2-bb28-1ec2a9b82e31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:13:35.372973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84053c7d-01f9-45b2-bb28-1ec2a9b82e31 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:35.373270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84053c7d-01f9-45b2-bb28-1ec2a9b82e31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:35.373550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84053c7d-01f9-45b2-bb28-1ec2a9b82e31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:35.375839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-84053c7d-01f9-45b2-bb28-1ec2a9b82e31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:35.498760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84053c7d-01f9-45b2-bb28-1ec2a9b82e31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:13:35.500262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84053c7d-01f9-45b2-bb28-1ec2a9b82e31 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:13:35.519250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 657/1331] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:13:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:13:38.196296 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee84f5c-0a79-4119-a95e-893cc2cbe44b None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:38.196827 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee84f5c-0a79-4119-a95e-893cc2cbe44b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:38.196827 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee84f5c-0a79-4119-a95e-893cc2cbe44b None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:38.198177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 675/1332] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:13:38.204272 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d01dd-c275-493d-af94-c169b92e9227 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:38.204687 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d01dd-c275-493d-af94-c169b92e9227 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:38.204833 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d01dd-c275-493d-af94-c169b92e9227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:38.268271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:38.270944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-e63d01dd-c275-493d-af94-c169b92e9227 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:13:38.272181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e63d01dd-c275-493d-af94-c169b92e9227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:38.369614 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 658/1333] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:13:38.406413 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee84f5c-0a79-4119-a95e-893cc2cbe44b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:13:38.407953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9ee84f5c-0a79-4119-a95e-893cc2cbe44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:38.511939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee84f5c-0a79-4119-a95e-893cc2cbe44b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:13:38.587262 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064e4e8a-f5b7-4e23-accd-609fa1798107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:13:38.587910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e4e8a-f5b7-4e23-accd-609fa1798107 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:38.588195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e4e8a-f5b7-4e23-accd-609fa1798107 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:38.588471 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064e4e8a-f5b7-4e23-accd-609fa1798107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:38.590161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-064e4e8a-f5b7-4e23-accd-609fa1798107 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:38.617671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:38.699650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e4e8a-f5b7-4e23-accd-609fa1798107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:13:38.701119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064e4e8a-f5b7-4e23-accd-609fa1798107 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:13:38.721303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 676/1334] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:13:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:13:40.928083 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130f788-ba06-430e-8f8e-be0c3994f280 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:40.928601 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130f788-ba06-430e-8f8e-be0c3994f280 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:40.928601 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130f788-ba06-430e-8f8e-be0c3994f280 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:40.929992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 659/1335] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:13:40.935154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190b483-7912-4c9d-b104-ddccce3fe475 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:40.935567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190b483-7912-4c9d-b104-ddccce3fe475 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:40.935707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190b483-7912-4c9d-b104-ddccce3fe475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:41.006249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-d190b483-7912-4c9d-b104-ddccce3fe475 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:13:41.007459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d190b483-7912-4c9d-b104-ddccce3fe475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:41.083911 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:41.106807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 677/1336] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:13:41.150107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130f788-ba06-430e-8f8e-be0c3994f280 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:13:41.151737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a130f788-ba06-430e-8f8e-be0c3994f280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:41.267278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130f788-ba06-430e-8f8e-be0c3994f280 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:13:41.297524 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:41.345613 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87873f-5b28-4a6f-9efe-fc11aaa030d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:13:41.346308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87873f-5b28-4a6f-9efe-fc11aaa030d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:41.346587 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87873f-5b28-4a6f-9efe-fc11aaa030d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:41.346864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87873f-5b28-4a6f-9efe-fc11aaa030d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:41.348521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1c87873f-5b28-4a6f-9efe-fc11aaa030d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:41.480190 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87873f-5b28-4a6f-9efe-fc11aaa030d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:13:41.481657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87873f-5b28-4a6f-9efe-fc11aaa030d1 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:13:41.501727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 660/1337] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:13:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:13:44.554852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d4230-5d75-4cc2-a69b-9805ef39fa62 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:44.555346 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d4230-5d75-4cc2-a69b-9805ef39fa62 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:44.555444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d4230-5d75-4cc2-a69b-9805ef39fa62 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:44.556808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 678/1338] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:13:44.562212 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dceac-9906-4f81-8d5d-f5c1a257b7b2 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:44.562688 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dceac-9906-4f81-8d5d-f5c1a257b7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:44.562883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1dceac-9906-4f81-8d5d-f5c1a257b7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:44.631011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-4d1dceac-9906-4f81-8d5d-f5c1a257b7b2 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:13:44.632227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4d1dceac-9906-4f81-8d5d-f5c1a257b7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:44.641217 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:44.728574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 661/1339] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:13:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:13:44.764014 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d4230-5d75-4cc2-a69b-9805ef39fa62 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:13:44.768175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2f6d4230-5d75-4cc2-a69b-9805ef39fa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:44.882440 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d4230-5d75-4cc2-a69b-9805ef39fa62 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:13:44.885959 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a4088-bb0d-4381-a65c-4b7e547e0725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:13:44.886607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a4088-bb0d-4381-a65c-4b7e547e0725 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:44.886967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a4088-bb0d-4381-a65c-4b7e547e0725 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:44.887250 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a4088-bb0d-4381-a65c-4b7e547e0725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:44.889113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d54a4088-bb0d-4381-a65c-4b7e547e0725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:44.991815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a4088-bb0d-4381-a65c-4b7e547e0725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:13:44.993354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a4088-bb0d-4381-a65c-4b7e547e0725 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:13:45.003917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:13:45.013075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 679/1340] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:13:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:13:49.656253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c665b-4fe4-4340-b5bf-61c58e0bb705 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:49.656813 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c665b-4fe4-4340-b5bf-61c58e0bb705 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:49.656813 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c665b-4fe4-4340-b5bf-61c58e0bb705 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:49.658235 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 662/1341] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:13:49.663697 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b99c9-9f58-4cd7-89dd-13eff958ddbb None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:49.664136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b99c9-9f58-4cd7-89dd-13eff958ddbb None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:49.664395 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b99c9-9f58-4cd7-89dd-13eff958ddbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:49.719077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-339b99c9-9f58-4cd7-89dd-13eff958ddbb None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:13:49.720506 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-339b99c9-9f58-4cd7-89dd-13eff958ddbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:49.803162 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 680/1342] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:13:49.842156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c665b-4fe4-4340-b5bf-61c58e0bb705 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:13:49.845075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ac8c665b-4fe4-4340-b5bf-61c58e0bb705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:49.859369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c665b-4fe4-4340-b5bf-61c58e0bb705 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:13:49.863060 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f716c9-14fe-4955-905c-ae459b28ba13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:13:49.863692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f716c9-14fe-4955-905c-ae459b28ba13 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:49.863972 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f716c9-14fe-4955-905c-ae459b28ba13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:49.864247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f716c9-14fe-4955-905c-ae459b28ba13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:49.873634 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f716c9-14fe-4955-905c-ae459b28ba13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:13:49.875407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f716c9-14fe-4955-905c-ae459b28ba13 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:13:49.885708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 663/1343] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:13:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:13:53.064980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e12092-9a88-46c7-b5e9-cf9f315ba85f None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:53.065507 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e12092-9a88-46c7-b5e9-cf9f315ba85f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:53.065617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e12092-9a88-46c7-b5e9-cf9f315ba85f None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:53.067138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 681/1344] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:13:53.073380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec88770-417b-47a2-9f89-ab434eef7841 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:53.073938 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec88770-417b-47a2-9f89-ab434eef7841 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:53.074057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec88770-417b-47a2-9f89-ab434eef7841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:53.104554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-9ec88770-417b-47a2-9f89-ab434eef7841 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:13:53.105763 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9ec88770-417b-47a2-9f89-ab434eef7841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:53.114437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 664/1345] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:13:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:13:53.152809 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e12092-9a88-46c7-b5e9-cf9f315ba85f None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:13:53.165331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e12092-9a88-46c7-b5e9-cf9f315ba85f None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:13:53.169071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509014d3-bd65-4949-a27d-17200581ebd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:13:53.169716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509014d3-bd65-4949-a27d-17200581ebd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:53.170007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509014d3-bd65-4949-a27d-17200581ebd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:53.170294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509014d3-bd65-4949-a27d-17200581ebd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:53.180409 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509014d3-bd65-4949-a27d-17200581ebd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:13:53.181893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509014d3-bd65-4949-a27d-17200581ebd1 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:13:53.193373 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 682/1346] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:13:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:13:57.893855 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e4cee-b041-4263-851e-08ac688a10d3 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:57.894405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e4cee-b041-4263-851e-08ac688a10d3 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:57.894405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e4cee-b041-4263-851e-08ac688a10d3 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:57.895825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 665/1347] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:13:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:13:57.901080 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db11eba-2b80-4110-905d-11c0032a9127 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:57.901514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db11eba-2b80-4110-905d-11c0032a9127 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:57.901636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db11eba-2b80-4110-905d-11c0032a9127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:57.923062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-9db11eba-2b80-4110-905d-11c0032a9127 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:13:57.924318 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9db11eba-2b80-4110-905d-11c0032a9127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:13:57.932563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 683/1348] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:13:57.975849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e4cee-b041-4263-851e-08ac688a10d3 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:13:57.995574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e4cee-b041-4263-851e-08ac688a10d3 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:13:57.999197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7b6ca-f7b1-4e25-9812-0bc9cb52887b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:13:57.999845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7b6ca-f7b1-4e25-9812-0bc9cb52887b service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:13:58.000131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7b6ca-f7b1-4e25-9812-0bc9cb52887b service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:13:58.000410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7b6ca-f7b1-4e25-9812-0bc9cb52887b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:13:58.010278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7b6ca-f7b1-4e25-9812-0bc9cb52887b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:13:58.011828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7b6ca-f7b1-4e25-9812-0bc9cb52887b service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:13:58.022685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 666/1349] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:13:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:14:03.156120 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91805d7-6b52-4484-990a-8312e2766f8d None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:03.157108 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91805d7-6b52-4484-990a-8312e2766f8d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:03.157396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91805d7-6b52-4484-990a-8312e2766f8d None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:03.158790 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 684/1350] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jun 15 10:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:03.164301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c6760-2511-4830-8950-3c4ecb1be00d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:03.164769 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c6760-2511-4830-8950-3c4ecb1be00d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:03.165149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c6760-2511-4830-8950-3c4ecb1be00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:03.274095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-742c6760-2511-4830-8950-3c4ecb1be00d None None] MFA Rules not processed for user `12899a7a59b54e3199b277ff94e30d42`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:03.275670 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-742c6760-2511-4830-8950-3c4ecb1be00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:03.324461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:14:03.354981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 667/1351] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jun 15 10:14:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:07.606222 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82528b0d-f5de-4219-9adc-1ed6ad063f26 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:07.607074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82528b0d-f5de-4219-9adc-1ed6ad063f26 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:07.607520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82528b0d-f5de-4219-9adc-1ed6ad063f26 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:07.609445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 685/1352] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jun 15 10:14:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:07.614621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e1a0e-0d32-4c74-99be-d81b4ed19937 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:07.615057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e1a0e-0d32-4c74-99be-d81b4ed19937 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:07.615411 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e1a0e-0d32-4c74-99be-d81b4ed19937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:07.635701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-3e3e1a0e-0d32-4c74-99be-d81b4ed19937 None None] MFA Rules not processed for user `12899a7a59b54e3199b277ff94e30d42`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:07.636913 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3e3e1a0e-0d32-4c74-99be-d81b4ed19937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:07.647023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 668/1353] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jun 15 10:14:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:07.915109 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=56 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:14:09.294790 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821dc5d-2f5f-4eb5-9bf7-eeb441bd6ddb None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:09.295569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821dc5d-2f5f-4eb5-9bf7-eeb441bd6ddb None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:09.295569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821dc5d-2f5f-4eb5-9bf7-eeb441bd6ddb None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:09.297082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 686/1354] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jun 15 10:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:09.302360 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73405972-bc2b-4a7f-a85b-08124f213162 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:09.302796 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73405972-bc2b-4a7f-a85b-08124f213162 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:09.302939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73405972-bc2b-4a7f-a85b-08124f213162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:09.322485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-73405972-bc2b-4a7f-a85b-08124f213162 None None] MFA Rules not processed for user `12899a7a59b54e3199b277ff94e30d42`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:09.323690 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-73405972-bc2b-4a7f-a85b-08124f213162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:09.333395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 669/1355] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jun 15 10:14:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:14:13.228770 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7ffd6-0a5a-4759-a74a-d7fb2666a4ba None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:13.229383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7ffd6-0a5a-4759-a74a-d7fb2666a4ba None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:13.229383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7ffd6-0a5a-4759-a74a-d7fb2666a4ba None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:13.230688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 687/1356] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:13.237097 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e3ac5-abcf-452f-ae1f-fa9d494e45f3 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:13.237561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e3ac5-abcf-452f-ae1f-fa9d494e45f3 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:13.237655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e3ac5-abcf-452f-ae1f-fa9d494e45f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:13.256503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-cc4e3ac5-abcf-452f-ae1f-fa9d494e45f3 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:13.257750 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-cc4e3ac5-abcf-452f-ae1f-fa9d494e45f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:13.267623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 670/1357] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:13.324421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae5d56-4a73-4340-a563-1d9f47ecd470 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:13.325448 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae5d56-4a73-4340-a563-1d9f47ecd470 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:13.325995 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae5d56-4a73-4340-a563-1d9f47ecd470 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:13.329332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 688/1358] 10.222.0.60 () {54 vars in 961 bytes} [Wed Jun 15 10:14:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:13.355050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0a215-20af-436b-9974-5bb4add93242 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:13.355310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0a215-20af-436b-9974-5bb4add93242 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:13.355533 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0a215-20af-436b-9974-5bb4add93242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:13.432741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-c7d0a215-20af-436b-9974-5bb4add93242 None None] MFA Rules not processed for user `26f23f8595fc43ad94aee7d0192164a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:13.436828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c7d0a215-20af-436b-9974-5bb4add93242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:13.542151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 671/1359] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Jun 15 10:14:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:13.551351 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae5d56-4a73-4340-a563-1d9f47ecd470 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:14:13.569750 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae5d56-4a73-4340-a563-1d9f47ecd470 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:14:13.576369 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3380d-793c-4e81-8449-4e9f75484db0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:14:13.577905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3380d-793c-4e81-8449-4e9f75484db0 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:13.578294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3380d-793c-4e81-8449-4e9f75484db0 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:13.578578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3380d-793c-4e81-8449-4e9f75484db0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:13.589877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3380d-793c-4e81-8449-4e9f75484db0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:14:13.591521 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3380d-793c-4e81-8449-4e9f75484db0 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:14:13.604847 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 689/1360] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:14:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:14:16.013632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6a27d-320c-4632-aaf5-23ea389e49c7 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:16.014718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6a27d-320c-4632-aaf5-23ea389e49c7 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:16.015292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb6a27d-320c-4632-aaf5-23ea389e49c7 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:16.017057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 672/1361] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:14:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:16.023322 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4185ad-2d2e-4399-ab6a-d5fb14bf7049 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:16.023815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4185ad-2d2e-4399-ab6a-d5fb14bf7049 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:16.023895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4185ad-2d2e-4399-ab6a-d5fb14bf7049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:16.046278 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-7a4185ad-2d2e-4399-ab6a-d5fb14bf7049 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:16.047474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7a4185ad-2d2e-4399-ab6a-d5fb14bf7049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:16.064389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 690/1362] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:16.110771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6a27d-320c-4632-aaf5-23ea389e49c7 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:14:16.136587 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb6a27d-320c-4632-aaf5-23ea389e49c7 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:14:16.144046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058daa32-d88d-4384-87fd-722ce141dfd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:14:16.144823 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058daa32-d88d-4384-87fd-722ce141dfd4 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:16.145242 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058daa32-d88d-4384-87fd-722ce141dfd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:16.145629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058daa32-d88d-4384-87fd-722ce141dfd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:16.155540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058daa32-d88d-4384-87fd-722ce141dfd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:14:16.157376 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058daa32-d88d-4384-87fd-722ce141dfd4 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:14:16.168536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 673/1363] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:14:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:14:18.866350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4df4e1-b57d-48bc-b03b-b2fd30f5fe9c None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:18.866825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4df4e1-b57d-48bc-b03b-b2fd30f5fe9c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:18.866946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4df4e1-b57d-48bc-b03b-b2fd30f5fe9c None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:18.868276 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 691/1364] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:18.874014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdebda-e71a-4b12-99c0-f0c6b0ed0fc9 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:18.874447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdebda-e71a-4b12-99c0-f0c6b0ed0fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:18.874621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdebda-e71a-4b12-99c0-f0c6b0ed0fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:18.893766 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-34bdebda-e71a-4b12-99c0-f0c6b0ed0fc9 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:18.895141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-34bdebda-e71a-4b12-99c0-f0c6b0ed0fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:18.904232 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 674/1365] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:14:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:18.959688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920307ca-49ea-43e7-96be-e0cb7c5dc20d None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:18.959972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920307ca-49ea-43e7-96be-e0cb7c5dc20d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:18.960248 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920307ca-49ea-43e7-96be-e0cb7c5dc20d None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:18.961698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 692/1366] 10.222.0.60 () {54 vars in 961 bytes} [Wed Jun 15 10:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:18.967642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cecb0-639c-4d37-91f0-185a5f26818a None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:18.968106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cecb0-639c-4d37-91f0-185a5f26818a None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:18.968385 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978cecb0-639c-4d37-91f0-185a5f26818a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:18.986902 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-978cecb0-639c-4d37-91f0-185a5f26818a None None] MFA Rules not processed for user `26f23f8595fc43ad94aee7d0192164a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:18.988010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-978cecb0-639c-4d37-91f0-185a5f26818a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:18.997167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 675/1367] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Jun 15 10:14:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:19.005248 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920307ca-49ea-43e7-96be-e0cb7c5dc20d None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:14:19.020856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920307ca-49ea-43e7-96be-e0cb7c5dc20d None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:14:19.026536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6294a4d3-2a3f-4269-88c6-104064656bb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:14:19.027183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294a4d3-2a3f-4269-88c6-104064656bb7 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:19.027466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294a4d3-2a3f-4269-88c6-104064656bb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:19.027741 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6294a4d3-2a3f-4269-88c6-104064656bb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:19.036702 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294a4d3-2a3f-4269-88c6-104064656bb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:14:19.038383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6294a4d3-2a3f-4269-88c6-104064656bb7 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:14:19.049981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 693/1368] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:14:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:14:21.554987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6447be4-b089-498f-b25e-cc10b9ce424c None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:21.555626 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6447be4-b089-498f-b25e-cc10b9ce424c None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:21.555626 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6447be4-b089-498f-b25e-cc10b9ce424c None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:21.557479 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 676/1369] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:21.563604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee716b63-658c-486e-99ef-68cd82933571 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:21.564133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee716b63-658c-486e-99ef-68cd82933571 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:21.564133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee716b63-658c-486e-99ef-68cd82933571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:21.583892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-ee716b63-658c-486e-99ef-68cd82933571 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:21.585154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ee716b63-658c-486e-99ef-68cd82933571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:21.597125 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 694/1370] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:21.637803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2b556-ba9a-4334-96a4-e68eeb543a19 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:21.638088 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2b556-ba9a-4334-96a4-e68eeb543a19 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:21.638385 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2b556-ba9a-4334-96a4-e68eeb543a19 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:21.639917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 677/1371] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:21.640498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee716b63-658c-486e-99ef-68cd82933571 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:14:21.644985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22183d6-c648-4012-ae5a-b0b0b3dcf70a None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:21.645286 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22183d6-c648-4012-ae5a-b0b0b3dcf70a None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:21.645566 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22183d6-c648-4012-ae5a-b0b0b3dcf70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:21.653813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee716b63-658c-486e-99ef-68cd82933571 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:14:21.659705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf52fa-235b-46fc-b95e-4a6aaa92929a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:14:21.660270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf52fa-235b-46fc-b95e-4a6aaa92929a service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:21.660554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf52fa-235b-46fc-b95e-4a6aaa92929a service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:21.660832 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf52fa-235b-46fc-b95e-4a6aaa92929a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:21.663881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b22183d6-c648-4012-ae5a-b0b0b3dcf70a None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:21.665200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b22183d6-c648-4012-ae5a-b0b0b3dcf70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:21.669800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf52fa-235b-46fc-b95e-4a6aaa92929a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:14:21.671941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf52fa-235b-46fc-b95e-4a6aaa92929a service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:14:21.681364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 678/1372] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:14:21.682619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 695/1373] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:14:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:14:21.991713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa0f2b-d46c-4545-9c6c-3d252915cbf3 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:21.992099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa0f2b-d46c-4545-9c6c-3d252915cbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:21.992445 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa0f2b-d46c-4545-9c6c-3d252915cbf3 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:21.994766 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 679/1374] 10.222.0.60 () {54 vars in 963 bytes} [Wed Jun 15 10:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:22.000071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f090a89-22ed-441a-b662-e55674bfc796 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:22.000469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f090a89-22ed-441a-b662-e55674bfc796 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:22.000814 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f090a89-22ed-441a-b662-e55674bfc796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:22.058586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-0f090a89-22ed-441a-b662-e55674bfc796 None None] MFA Rules not processed for user `fa51162a69d44127bd34df2f4653dc15`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:22.059797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0f090a89-22ed-441a-b662-e55674bfc796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:22.129300 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 696/1375] 10.222.0.60 () {58 vars in 1060 bytes} [Wed Jun 15 10:14:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:22.137253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa0f2b-d46c-4545-9c6c-3d252915cbf3 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:14:22.149460 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa0f2b-d46c-4545-9c6c-3d252915cbf3 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:14:22.154360 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af372f36-73a1-4f79-8943-a4bfe1ecebd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:14:22.155621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af372f36-73a1-4f79-8943-a4bfe1ecebd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:22.155976 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af372f36-73a1-4f79-8943-a4bfe1ecebd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:22.156320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af372f36-73a1-4f79-8943-a4bfe1ecebd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:22.166977 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af372f36-73a1-4f79-8943-a4bfe1ecebd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:14:22.168546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af372f36-73a1-4f79-8943-a4bfe1ecebd7 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:14:22.179917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 680/1376] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:14:24.040572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb48c8f-30bd-44ad-a333-489e74d369bb None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:24.041138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb48c8f-30bd-44ad-a333-489e74d369bb None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:24.041246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb48c8f-30bd-44ad-a333-489e74d369bb None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:24.042563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 697/1377] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:24.048398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcddb8f-3b84-4bd0-9562-67f2d44addbb None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:24.048841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcddb8f-3b84-4bd0-9562-67f2d44addbb None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:24.048998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcddb8f-3b84-4bd0-9562-67f2d44addbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:24.068012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-2dcddb8f-3b84-4bd0-9562-67f2d44addbb None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:24.069175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-2dcddb8f-3b84-4bd0-9562-67f2d44addbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:24.077312 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 681/1378] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:14:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:24.123452 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb48c8f-30bd-44ad-a333-489e74d369bb None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:14:24.138908 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb48c8f-30bd-44ad-a333-489e74d369bb None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:14:24.144498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b0a73-60c9-462f-bd1c-ac16325edc8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:14:24.145167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b0a73-60c9-462f-bd1c-ac16325edc8c service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:24.145452 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b0a73-60c9-462f-bd1c-ac16325edc8c service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:24.145729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b0a73-60c9-462f-bd1c-ac16325edc8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:24.155368 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b0a73-60c9-462f-bd1c-ac16325edc8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:14:24.157025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b0a73-60c9-462f-bd1c-ac16325edc8c service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:14:24.167134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 698/1379] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:14:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:14:26.571376 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2c98f-da51-4fc5-a114-004eae065de0 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:26.571926 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2c98f-da51-4fc5-a114-004eae065de0 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:26.572200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c2c98f-da51-4fc5-a114-004eae065de0 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:26.573827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 682/1380] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:14:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:26.579623 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26bdc9-ac01-4ef5-87ff-98ceb2447e08 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:26.580065 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26bdc9-ac01-4ef5-87ff-98ceb2447e08 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:26.580172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26bdc9-ac01-4ef5-87ff-98ceb2447e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:26.599493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-af26bdc9-ac01-4ef5-87ff-98ceb2447e08 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:26.600685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-af26bdc9-ac01-4ef5-87ff-98ceb2447e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:26.609039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 699/1381] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:26.656492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2c98f-da51-4fc5-a114-004eae065de0 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:14:26.672082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c2c98f-da51-4fc5-a114-004eae065de0 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:14:26.677741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ce049-63d9-4824-9aa1-c81474a443b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:14:26.678552 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ce049-63d9-4824-9aa1-c81474a443b9 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:26.678944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ce049-63d9-4824-9aa1-c81474a443b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:26.679326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ce049-63d9-4824-9aa1-c81474a443b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:26.688853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ce049-63d9-4824-9aa1-c81474a443b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:14:26.690610 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ce049-63d9-4824-9aa1-c81474a443b9 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:14:26.701670 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 683/1382] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:14:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:14:29.058674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b91033-6230-4a5b-8680-75e33ee8085b None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:29.059196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b91033-6230-4a5b-8680-75e33ee8085b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:29.059196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b91033-6230-4a5b-8680-75e33ee8085b None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:29.061415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 700/1383] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:14:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:29.067164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991287c-ea80-4170-9d0a-4bd3166858d2 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:29.067907 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991287c-ea80-4170-9d0a-4bd3166858d2 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:29.067994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991287c-ea80-4170-9d0a-4bd3166858d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:29.075076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:14:29.087557 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-3991287c-ea80-4170-9d0a-4bd3166858d2 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:29.088986 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3991287c-ea80-4170-9d0a-4bd3166858d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:29.103420 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 684/1384] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:14:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:29.141941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b91033-6230-4a5b-8680-75e33ee8085b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:14:29.157572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b91033-6230-4a5b-8680-75e33ee8085b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:14:29.163532 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81865fb2-ffb1-4332-b21d-2a44bdf79da9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:14:29.164321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81865fb2-ffb1-4332-b21d-2a44bdf79da9 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:29.164754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81865fb2-ffb1-4332-b21d-2a44bdf79da9 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:29.165163 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81865fb2-ffb1-4332-b21d-2a44bdf79da9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:29.177634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81865fb2-ffb1-4332-b21d-2a44bdf79da9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:14:29.179350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81865fb2-ffb1-4332-b21d-2a44bdf79da9 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:14:29.190848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 701/1385] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:14:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:14:31.553786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32106d4-3e44-4834-b0d9-a773ae4465f9 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:31.554273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32106d4-3e44-4834-b0d9-a773ae4465f9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:31.554368 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32106d4-3e44-4834-b0d9-a773ae4465f9 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:31.556009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 685/1386] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:14:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:31.563227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80afbc-4621-45db-ade8-db8b44ae2cff None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:31.564036 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80afbc-4621-45db-ade8-db8b44ae2cff None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:31.564036 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80afbc-4621-45db-ade8-db8b44ae2cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:31.584432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-0d80afbc-4621-45db-ade8-db8b44ae2cff None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:31.585921 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0d80afbc-4621-45db-ade8-db8b44ae2cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:31.596052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 702/1387] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:31.641811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32106d4-3e44-4834-b0d9-a773ae4465f9 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:14:31.661064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32106d4-3e44-4834-b0d9-a773ae4465f9 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:14:31.667876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4934d-d12f-4b4b-967d-a38fcfb64dae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:14:31.668741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4934d-d12f-4b4b-967d-a38fcfb64dae service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:31.669146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4934d-d12f-4b4b-967d-a38fcfb64dae service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:31.669515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4934d-d12f-4b4b-967d-a38fcfb64dae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:31.681330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4934d-d12f-4b4b-967d-a38fcfb64dae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:14:31.683634 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4934d-d12f-4b4b-967d-a38fcfb64dae service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:14:31.695606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 686/1388] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:14:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:14:34.088751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f327d-90ff-423a-bd7a-03bda7ba8ec6 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:34.089416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f327d-90ff-423a-bd7a-03bda7ba8ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:34.089416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f327d-90ff-423a-bd7a-03bda7ba8ec6 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:34.090671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 703/1389] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:34.098788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee721a-9cd7-4916-9590-0f57368def9a None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:34.099489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee721a-9cd7-4916-9590-0f57368def9a None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:34.099687 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee721a-9cd7-4916-9590-0f57368def9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:34.119307 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a7ee721a-9cd7-4916-9590-0f57368def9a None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:34.120530 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a7ee721a-9cd7-4916-9590-0f57368def9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:34.128814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 687/1390] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:34.176028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f327d-90ff-423a-bd7a-03bda7ba8ec6 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:14:34.199218 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f327d-90ff-423a-bd7a-03bda7ba8ec6 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:14:34.208803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3ad29-b645-444f-a2cd-f61e77bbd0dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:14:34.209830 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3ad29-b645-444f-a2cd-f61e77bbd0dd service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:34.210381 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3ad29-b645-444f-a2cd-f61e77bbd0dd service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:34.210812 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3ad29-b645-444f-a2cd-f61e77bbd0dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:34.224653 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f3ad29-b645-444f-a2cd-f61e77bbd0dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:14:34.227237 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f3ad29-b645-444f-a2cd-f61e77bbd0dd service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:14:34.243767 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 704/1391] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:14:36.683757 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367bfa0-cdb5-4b7a-bcd7-df555ddce9d7 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:36.684305 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367bfa0-cdb5-4b7a-bcd7-df555ddce9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:36.684395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367bfa0-cdb5-4b7a-bcd7-df555ddce9d7 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:36.686308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 688/1392] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:36.692069 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f519d4-fd50-4921-9219-f26cd5c6af7c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:36.692876 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f519d4-fd50-4921-9219-f26cd5c6af7c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:36.693008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f519d4-fd50-4921-9219-f26cd5c6af7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:36.711189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-09f519d4-fd50-4921-9219-f26cd5c6af7c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:36.712391 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-09f519d4-fd50-4921-9219-f26cd5c6af7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:36.720834 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 705/1393] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:36.765804 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367bfa0-cdb5-4b7a-bcd7-df555ddce9d7 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:14:36.780650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367bfa0-cdb5-4b7a-bcd7-df555ddce9d7 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:14:36.786078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5eb60e-ae77-4c73-88b1-8d71d2c3c777 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:14:36.786731 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5eb60e-ae77-4c73-88b1-8d71d2c3c777 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:36.787140 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5eb60e-ae77-4c73-88b1-8d71d2c3c777 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:36.787415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5eb60e-ae77-4c73-88b1-8d71d2c3c777 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:36.797052 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5eb60e-ae77-4c73-88b1-8d71d2c3c777 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:14:36.798563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5eb60e-ae77-4c73-88b1-8d71d2c3c777 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:14:36.808718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 689/1394] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:14:38.512774 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b0730-38bd-492c-94f2-a460268d45cf None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:38.513400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b0730-38bd-492c-94f2-a460268d45cf None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:38.513400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b0730-38bd-492c-94f2-a460268d45cf None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:38.514802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 706/1395] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:38.520228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21550dce-df81-49e3-a263-38165a6cbb60 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:38.520649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21550dce-df81-49e3-a263-38165a6cbb60 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:38.520798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21550dce-df81-49e3-a263-38165a6cbb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:38.539675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-21550dce-df81-49e3-a263-38165a6cbb60 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:38.540873 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-21550dce-df81-49e3-a263-38165a6cbb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:38.549366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 690/1396] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:38.606059 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe3b50-6f19-44ef-a706-ee12da88f2f0 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:38.606484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe3b50-6f19-44ef-a706-ee12da88f2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:38.606806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe3b50-6f19-44ef-a706-ee12da88f2f0 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:38.608760 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 707/1397] 10.222.0.60 () {54 vars in 962 bytes} [Wed Jun 15 10:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:38.614251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f385c-ac93-4cee-a58d-1ffae1efce8e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:38.614546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f385c-ac93-4cee-a58d-1ffae1efce8e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:38.614829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f385c-ac93-4cee-a58d-1ffae1efce8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:38.667481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-d54f385c-ac93-4cee-a58d-1ffae1efce8e None None] MFA Rules not processed for user `7fbf33189aa94f4f973d558d615a23e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:38.668678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d54f385c-ac93-4cee-a58d-1ffae1efce8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:38.729917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 691/1398] 10.222.0.60 () {58 vars in 1059 bytes} [Wed Jun 15 10:14:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:38.738401 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe3b50-6f19-44ef-a706-ee12da88f2f0 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:14:38.743877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=90 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:14:38.750018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fe3b50-6f19-44ef-a706-ee12da88f2f0 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:14:38.753726 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b2e43-90a1-4f23-9fcf-49704a9f93a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fbf33189aa94f4f973d558d615a23e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:14:38.754362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b2e43-90a1-4f23-9fcf-49704a9f93a5 service glance] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:38.754645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b2e43-90a1-4f23-9fcf-49704a9f93a5 service glance] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:38.754913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b2e43-90a1-4f23-9fcf-49704a9f93a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:38.764736 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b2e43-90a1-4f23-9fcf-49704a9f93a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:14:38.766148 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b2e43-90a1-4f23-9fcf-49704a9f93a5 service glance] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:14:38.776046 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 708/1399] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:14:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:14:39.181984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f07fa8-7f62-4cc4-ab2a-663c4e5e3e4e None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:39.182283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f07fa8-7f62-4cc4-ab2a-663c4e5e3e4e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:39.182565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f07fa8-7f62-4cc4-ab2a-663c4e5e3e4e None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:39.183993 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 692/1400] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:14:39.190434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb9132-e909-4b8d-8938-4405e0d990e7 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:39.190824 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb9132-e909-4b8d-8938-4405e0d990e7 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:39.191114 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb9132-e909-4b8d-8938-4405e0d990e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:39.209699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-1afb9132-e909-4b8d-8938-4405e0d990e7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:39.210972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-1afb9132-e909-4b8d-8938-4405e0d990e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:39.219834 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 709/1401] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:39.276887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f07fa8-7f62-4cc4-ab2a-663c4e5e3e4e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:14:39.297817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f07fa8-7f62-4cc4-ab2a-663c4e5e3e4e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:14:39.305229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e38a15-546d-43fe-b0fa-cf480796f457 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:14:39.306565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e38a15-546d-43fe-b0fa-cf480796f457 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:39.307008 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e38a15-546d-43fe-b0fa-cf480796f457 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:39.307432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e38a15-546d-43fe-b0fa-cf480796f457 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:39.320585 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e38a15-546d-43fe-b0fa-cf480796f457 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:14:39.322375 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e38a15-546d-43fe-b0fa-cf480796f457 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:14:39.334584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 693/1402] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:14:39.726430 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e15305-1685-41f5-835d-8d9aece1ecf0 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:39.726911 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e15305-1685-41f5-835d-8d9aece1ecf0 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:39.727017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e15305-1685-41f5-835d-8d9aece1ecf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:39.778466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-b3e15305-1685-41f5-835d-8d9aece1ecf0 None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:39.780165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b3e15305-1685-41f5-835d-8d9aece1ecf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:39.848449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 710/1403] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jun 15 10:14:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:39.895208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c33f90-2121-4c77-883d-4dfb3da42318 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:39.895726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c33f90-2121-4c77-883d-4dfb3da42318 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:39.895828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c33f90-2121-4c77-883d-4dfb3da42318 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:39.897230 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 694/1404] 10.222.0.60 () {54 vars in 942 bytes} [Wed Jun 15 10:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:39.903859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401d9a8-547d-4b91-a4f7-223f935719b9 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:39.904762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401d9a8-547d-4b91-a4f7-223f935719b9 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:39.905081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4401d9a8-547d-4b91-a4f7-223f935719b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:39.957574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-4401d9a8-547d-4b91-a4f7-223f935719b9 None None] MFA Rules not processed for user `589e66c6228a44adbc9e418794d7d91b`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:39.958787 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-4401d9a8-547d-4b91-a4f7-223f935719b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:40.041887 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 711/1405] 10.222.0.60 () {58 vars in 1039 bytes} [Wed Jun 15 10:14:39 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:40.052745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c33f90-2121-4c77-883d-4dfb3da42318 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:14:40.067221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c33f90-2121-4c77-883d-4dfb3da42318 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:14:40.072626 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c222d2-2e9a-4b0a-9783-879180315727 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:14:40.073452 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c222d2-2e9a-4b0a-9783-879180315727 service swift] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:40.073691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c222d2-2e9a-4b0a-9783-879180315727 service swift] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:40.073971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c222d2-2e9a-4b0a-9783-879180315727 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:40.082756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c222d2-2e9a-4b0a-9783-879180315727 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:14:40.084336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c222d2-2e9a-4b0a-9783-879180315727 service swift] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:14:40.092502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 695/1406] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 10:14:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:14:41.708350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb539e-8e99-4edd-ac1d-f81301789cdb None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:41.708904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb539e-8e99-4edd-ac1d-f81301789cdb None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:41.708904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb539e-8e99-4edd-ac1d-f81301789cdb None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:41.710324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 712/1407] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:14:41.715560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0b0b3-a5fe-41a4-b5b9-bcaa9df12fc1 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:41.716038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0b0b3-a5fe-41a4-b5b9-bcaa9df12fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:41.716135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0b0b3-a5fe-41a4-b5b9-bcaa9df12fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:41.738369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-f7d0b0b3-a5fe-41a4-b5b9-bcaa9df12fc1 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:14:41.739599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f7d0b0b3-a5fe-41a4-b5b9-bcaa9df12fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:14:41.749873 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 696/1408] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:14:41.797536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb539e-8e99-4edd-ac1d-f81301789cdb None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:14:41.818024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb539e-8e99-4edd-ac1d-f81301789cdb None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:14:41.844541 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88346dac-d926-48e7-be95-52bfea44fa06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:14:41.845463 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88346dac-d926-48e7-be95-52bfea44fa06 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:14:41.845819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88346dac-d926-48e7-be95-52bfea44fa06 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:14:41.846253 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88346dac-d926-48e7-be95-52bfea44fa06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:14:41.865398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88346dac-d926-48e7-be95-52bfea44fa06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:14:41.867839 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88346dac-d926-48e7-be95-52bfea44fa06 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:14:41.892420 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 713/1409] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:14:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:14:51.729156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=72 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:14:51.874402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=98 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:15:05.296876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcf13a-0b63-45dc-b3f8-a8edf1e05946 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:05.297542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcf13a-0b63-45dc-b3f8-a8edf1e05946 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:05.297542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcf13a-0b63-45dc-b3f8-a8edf1e05946 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:05.298875 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 697/1410] 10.222.0.60 () {54 vars in 918 bytes} [Wed Jun 15 10:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:05.304097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc037c9-f783-42b5-a200-5b78f913c069 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:05.304572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc037c9-f783-42b5-a200-5b78f913c069 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:05.304679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc037c9-f783-42b5-a200-5b78f913c069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:05.322682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-7dc037c9-f783-42b5-a200-5b78f913c069 None None] MFA Rules not processed for user `12899a7a59b54e3199b277ff94e30d42`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:05.323956 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7dc037c9-f783-42b5-a200-5b78f913c069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:05.333038 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 714/1411] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Jun 15 10:15:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:05.348149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499f33e-96e1-4174-ab87-b30cd592e0ed None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:05.348571 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499f33e-96e1-4174-ab87-b30cd592e0ed None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:05.348702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499f33e-96e1-4174-ab87-b30cd592e0ed None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:05.350140 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 698/1412] 10.222.0.60 () {54 vars in 918 bytes} [Wed Jun 15 10:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:15:05.354792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d1592-c138-486f-a915-c3a7c34ba9f8 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:05.355079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d1592-c138-486f-a915-c3a7c34ba9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:05.355353 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d1592-c138-486f-a915-c3a7c34ba9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:05.373785 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-817d1592-c138-486f-a915-c3a7c34ba9f8 None None] MFA Rules not processed for user `26f23f8595fc43ad94aee7d0192164a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:05.375009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-817d1592-c138-486f-a915-c3a7c34ba9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:05.384600 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 715/1413] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Jun 15 10:15:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:08.015866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc017bf5-460b-4d74-a191-2a500c156fc0 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:08.016433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc017bf5-460b-4d74-a191-2a500c156fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:08.016433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc017bf5-460b-4d74-a191-2a500c156fc0 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:08.017819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:08.023283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215b46f-c098-4545-aab6-ee31876c6760 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:08.023738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215b46f-c098-4545-aab6-ee31876c6760 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:08.023841 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215b46f-c098-4545-aab6-ee31876c6760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:08.042132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-5215b46f-c098-4545-aab6-ee31876c6760 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:08.043314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5215b46f-c098-4545-aab6-ee31876c6760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:08.054080 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:08.098575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc017bf5-460b-4d74-a191-2a500c156fc0 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:08.113528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc017bf5-460b-4d74-a191-2a500c156fc0 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:08.118981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf765fd-ad41-4999-8e65-c762fcb56093 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:08.119613 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf765fd-ad41-4999-8e65-c762fcb56093 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:08.119895 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf765fd-ad41-4999-8e65-c762fcb56093 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:08.120169 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf765fd-ad41-4999-8e65-c762fcb56093 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:08.128476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf765fd-ad41-4999-8e65-c762fcb56093 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:08.130113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf765fd-ad41-4999-8e65-c762fcb56093 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:08.139791 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 700/1416] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:09.111400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe2318-aabd-47ea-a330-067ad505b713 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:09.111917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe2318-aabd-47ea-a330-067ad505b713 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:09.111917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe2318-aabd-47ea-a330-067ad505b713 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:09.113291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:09.118255 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c6864-39ae-4f64-80bb-ee0e8e688945 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:09.118707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c6864-39ae-4f64-80bb-ee0e8e688945 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:09.118829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c6864-39ae-4f64-80bb-ee0e8e688945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:09.137584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-0f7c6864-39ae-4f64-80bb-ee0e8e688945 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:09.139438 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-0f7c6864-39ae-4f64-80bb-ee0e8e688945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:09.149063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:09.315443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe2318-aabd-47ea-a330-067ad505b713 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:09.331018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe2318-aabd-47ea-a330-067ad505b713 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:09.336379 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17542231-964e-437d-8c3c-f1cc3df0e031 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:09.337039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17542231-964e-437d-8c3c-f1cc3df0e031 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:09.337321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17542231-964e-437d-8c3c-f1cc3df0e031 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:09.337597 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17542231-964e-437d-8c3c-f1cc3df0e031 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:09.346833 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17542231-964e-437d-8c3c-f1cc3df0e031 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:09.348404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17542231-964e-437d-8c3c-f1cc3df0e031 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:09.358264 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 718/1419] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:10.302583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726049dd-5ee5-46f6-a3ff-4f897e70ad42 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:10.303327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726049dd-5ee5-46f6-a3ff-4f897e70ad42 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:10.303327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726049dd-5ee5-46f6-a3ff-4f897e70ad42 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:10.304529 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:10.310029 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da0185-dd6f-45df-a742-033dd6056e55 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:10.310471 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da0185-dd6f-45df-a742-033dd6056e55 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:10.310726 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da0185-dd6f-45df-a742-033dd6056e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:10.328783 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-44da0185-dd6f-45df-a742-033dd6056e55 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:10.330165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-44da0185-dd6f-45df-a742-033dd6056e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:10.338071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:10.493609 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726049dd-5ee5-46f6-a3ff-4f897e70ad42 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:10.507742 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726049dd-5ee5-46f6-a3ff-4f897e70ad42 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:10.514140 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe555dc-5c1b-454e-9112-88b9df566523 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:10.514769 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe555dc-5c1b-454e-9112-88b9df566523 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:10.515066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe555dc-5c1b-454e-9112-88b9df566523 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:10.515342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe555dc-5c1b-454e-9112-88b9df566523 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:10.523660 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe555dc-5c1b-454e-9112-88b9df566523 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:10.525271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe555dc-5c1b-454e-9112-88b9df566523 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:10.536029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 703/1422] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:11.458659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e782c-53c7-4897-8970-6f3802a9e763 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:11.459255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e782c-53c7-4897-8970-6f3802a9e763 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:11.459255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e782c-53c7-4897-8970-6f3802a9e763 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:11.460656 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:11.466348 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8a4ba-6419-4c04-9f82-cd993e7655a0 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:11.466837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8a4ba-6419-4c04-9f82-cd993e7655a0 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:11.467011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8a4ba-6419-4c04-9f82-cd993e7655a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:11.485114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-34b8a4ba-6419-4c04-9f82-cd993e7655a0 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:11.493482 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-34b8a4ba-6419-4c04-9f82-cd993e7655a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:11.494332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:11.649671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e782c-53c7-4897-8970-6f3802a9e763 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:11.664178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e782c-53c7-4897-8970-6f3802a9e763 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:11.670901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0b4d6-487e-4e36-928d-43b367ba4165 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:11.671698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0b4d6-487e-4e36-928d-43b367ba4165 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:11.676421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0b4d6-487e-4e36-928d-43b367ba4165 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:11.676421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0b4d6-487e-4e36-928d-43b367ba4165 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:11.680684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0b4d6-487e-4e36-928d-43b367ba4165 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:11.682358 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e0b4d6-487e-4e36-928d-43b367ba4165 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:11.692457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 721/1425] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:12.614278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0e373-9841-48ae-ae26-28bff06360c3 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:12.614830 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0e373-9841-48ae-ae26-28bff06360c3 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:12.614830 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0e373-9841-48ae-ae26-28bff06360c3 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:12.616196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:12.621071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e3bd3-793d-454e-8692-ca81ca91d1e5 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:12.621473 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e3bd3-793d-454e-8692-ca81ca91d1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:12.621635 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e3bd3-793d-454e-8692-ca81ca91d1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:12.640521 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-562e3bd3-793d-454e-8692-ca81ca91d1e5 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:12.648258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-562e3bd3-793d-454e-8692-ca81ca91d1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:12.649754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:12.801931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0e373-9841-48ae-ae26-28bff06360c3 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:12.816900 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0e373-9841-48ae-ae26-28bff06360c3 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:12.822399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f49ae-48d6-42a5-91a7-dd5f07babc33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:12.823025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f49ae-48d6-42a5-91a7-dd5f07babc33 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:12.823306 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f49ae-48d6-42a5-91a7-dd5f07babc33 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:12.823581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f49ae-48d6-42a5-91a7-dd5f07babc33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:12.831965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f49ae-48d6-42a5-91a7-dd5f07babc33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:12.833561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f49ae-48d6-42a5-91a7-dd5f07babc33 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:12.843205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 706/1428] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:13.777433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65eebb2-7cea-4bbf-88cc-8e83e6b6b615 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:13.783938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65eebb2-7cea-4bbf-88cc-8e83e6b6b615 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:13.783938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65eebb2-7cea-4bbf-88cc-8e83e6b6b615 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:13.783938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:15:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:13.785110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a541232-099e-425d-be49-8bdb9b1cabc0 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:13.785506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a541232-099e-425d-be49-8bdb9b1cabc0 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:13.785672 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a541232-099e-425d-be49-8bdb9b1cabc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:13.803447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-7a541232-099e-425d-be49-8bdb9b1cabc0 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:13.804627 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-7a541232-099e-425d-be49-8bdb9b1cabc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:13.813224 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:13.968355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65eebb2-7cea-4bbf-88cc-8e83e6b6b615 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:13.983118 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65eebb2-7cea-4bbf-88cc-8e83e6b6b615 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:13.988474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7654e1-5f51-458e-bb63-d11f59391e6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:13.989134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7654e1-5f51-458e-bb63-d11f59391e6d service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:13.989462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7654e1-5f51-458e-bb63-d11f59391e6d service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:13.989733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7654e1-5f51-458e-bb63-d11f59391e6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:13.998511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7654e1-5f51-458e-bb63-d11f59391e6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:14.000058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7654e1-5f51-458e-bb63-d11f59391e6d service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:14.009631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 724/1431] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:14.966769 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a269f10-8204-4a2b-9b6f-d2500419731c None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:14.967347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a269f10-8204-4a2b-9b6f-d2500419731c None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:14.967347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a269f10-8204-4a2b-9b6f-d2500419731c None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:14.968685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:15:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:14.973592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d7ad2-b7a8-40c7-aba4-302a87964583 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:14.974048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d7ad2-b7a8-40c7-aba4-302a87964583 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:14.974161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d7ad2-b7a8-40c7-aba4-302a87964583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:14.993681 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-991d7ad2-b7a8-40c7-aba4-302a87964583 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:14.994878 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-991d7ad2-b7a8-40c7-aba4-302a87964583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:15.003216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:15.159725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a269f10-8204-4a2b-9b6f-d2500419731c None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:15.174511 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a269f10-8204-4a2b-9b6f-d2500419731c None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:15.179899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63bd1cc-894f-4ee1-8810-d4488ac480fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:15.182461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bd1cc-894f-4ee1-8810-d4488ac480fb service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:15.182461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bd1cc-894f-4ee1-8810-d4488ac480fb service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:15.182461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63bd1cc-894f-4ee1-8810-d4488ac480fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:15.190075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63bd1cc-894f-4ee1-8810-d4488ac480fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:15.191708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63bd1cc-894f-4ee1-8810-d4488ac480fb service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:15.203250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 709/1434] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:16.181847 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da912d71-8de0-4542-acb0-6b7d20fa3057 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:16.182382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da912d71-8de0-4542-acb0-6b7d20fa3057 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:16.182505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da912d71-8de0-4542-acb0-6b7d20fa3057 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:16.183775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:16.189509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7538c-65d2-4e10-adc9-7e392ebe4c64 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:16.189964 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7538c-65d2-4e10-adc9-7e392ebe4c64 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:16.190064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7538c-65d2-4e10-adc9-7e392ebe4c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:16.209320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b2a7538c-65d2-4e10-adc9-7e392ebe4c64 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:16.210556 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b2a7538c-65d2-4e10-adc9-7e392ebe4c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:16.218749 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:16.375522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da912d71-8de0-4542-acb0-6b7d20fa3057 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:16.390204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da912d71-8de0-4542-acb0-6b7d20fa3057 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:16.395644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81515a2d-d0d8-4215-9ae2-81364deb4fb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:16.396335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81515a2d-d0d8-4215-9ae2-81364deb4fb8 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:16.396640 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81515a2d-d0d8-4215-9ae2-81364deb4fb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:16.396901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81515a2d-d0d8-4215-9ae2-81364deb4fb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:16.405821 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81515a2d-d0d8-4215-9ae2-81364deb4fb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:16.407397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81515a2d-d0d8-4215-9ae2-81364deb4fb8 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:16.417000 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 727/1437] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:17.415944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c3267-3daa-4cd6-a69c-8f160f77d008 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:17.416477 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c3267-3daa-4cd6-a69c-8f160f77d008 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:17.416477 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c3267-3daa-4cd6-a69c-8f160f77d008 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:17.417831 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:17.423480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f569c4-0629-4d98-b871-16d7565dafaa None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:17.423891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f569c4-0629-4d98-b871-16d7565dafaa None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:17.424038 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f569c4-0629-4d98-b871-16d7565dafaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:17.442424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-59f569c4-0629-4d98-b871-16d7565dafaa None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:17.443868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-59f569c4-0629-4d98-b871-16d7565dafaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:17.454500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:17.604560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c3267-3daa-4cd6-a69c-8f160f77d008 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:17.619448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c3267-3daa-4cd6-a69c-8f160f77d008 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:17.625550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e549a-701c-4f0e-953b-5f855dc7cd7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:17.626184 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e549a-701c-4f0e-953b-5f855dc7cd7c service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:17.626470 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e549a-701c-4f0e-953b-5f855dc7cd7c service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:17.626748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e549a-701c-4f0e-953b-5f855dc7cd7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:17.634903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e549a-701c-4f0e-953b-5f855dc7cd7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:17.636501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e549a-701c-4f0e-953b-5f855dc7cd7c service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:17.646269 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 712/1440] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:19.734609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dba10-bdff-431f-bdc8-cf2fc5cea5ed None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:19.735146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dba10-bdff-431f-bdc8-cf2fc5cea5ed None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:19.735146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dba10-bdff-431f-bdc8-cf2fc5cea5ed None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:19.736541 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:19.742277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae7039-bcb6-4883-a70c-f94939be1569 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:19.742711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae7039-bcb6-4883-a70c-f94939be1569 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:19.742844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ae7039-bcb6-4883-a70c-f94939be1569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:19.761835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-49ae7039-bcb6-4883-a70c-f94939be1569 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:19.763053 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-49ae7039-bcb6-4883-a70c-f94939be1569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:19.771364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:19.811243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dba10-bdff-431f-bdc8-cf2fc5cea5ed None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:19.822300 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dba10-bdff-431f-bdc8-cf2fc5cea5ed None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:19.825978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda18c6e-f474-42b1-9011-eccedc518e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:19.826626 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda18c6e-f474-42b1-9011-eccedc518e96 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:19.826909 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda18c6e-f474-42b1-9011-eccedc518e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:19.827188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda18c6e-f474-42b1-9011-eccedc518e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:19.836654 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda18c6e-f474-42b1-9011-eccedc518e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:19.838156 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda18c6e-f474-42b1-9011-eccedc518e96 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:19.847784 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 730/1443] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:22.174360 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b927b1d-ae2e-4a3c-a6d3-4402305241a3 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:22.174948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b927b1d-ae2e-4a3c-a6d3-4402305241a3 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:22.174948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b927b1d-ae2e-4a3c-a6d3-4402305241a3 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:22.176287 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:22.181680 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16612a42-0bec-4288-be7e-3ad6c5b9d48d None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:22.182061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16612a42-0bec-4288-be7e-3ad6c5b9d48d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:22.182235 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16612a42-0bec-4288-be7e-3ad6c5b9d48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:22.201397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-16612a42-0bec-4288-be7e-3ad6c5b9d48d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:22.202586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-16612a42-0bec-4288-be7e-3ad6c5b9d48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:22.210493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:22.249414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b927b1d-ae2e-4a3c-a6d3-4402305241a3 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:22.260004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b927b1d-ae2e-4a3c-a6d3-4402305241a3 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:22.263528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1bf551-94ee-4c08-90de-484a2e428a9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:22.264162 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bf551-94ee-4c08-90de-484a2e428a9e service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:22.264468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bf551-94ee-4c08-90de-484a2e428a9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:22.264744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bf551-94ee-4c08-90de-484a2e428a9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:22.274969 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1bf551-94ee-4c08-90de-484a2e428a9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:22.276424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1bf551-94ee-4c08-90de-484a2e428a9e service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:22.286520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 715/1446] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:24.909196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98a4f4-bf4f-49df-8e20-db57dc4ae9d6 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:24.909841 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98a4f4-bf4f-49df-8e20-db57dc4ae9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:24.909841 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98a4f4-bf4f-49df-8e20-db57dc4ae9d6 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:24.911146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:24.916633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b0580-effb-43e8-9f26-da35d75d7ee3 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:24.917111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b0580-effb-43e8-9f26-da35d75d7ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:24.917238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b0580-effb-43e8-9f26-da35d75d7ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:24.935685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-1e2b0580-effb-43e8-9f26-da35d75d7ee3 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:24.936876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1e2b0580-effb-43e8-9f26-da35d75d7ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:24.944810 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:24.984334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98a4f4-bf4f-49df-8e20-db57dc4ae9d6 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:24.995048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98a4f4-bf4f-49df-8e20-db57dc4ae9d6 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:24.999077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd98fa-bfa1-4ba4-a27d-5a382ece0b3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:24.999736 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd98fa-bfa1-4ba4-a27d-5a382ece0b3f service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:25.000015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd98fa-bfa1-4ba4-a27d-5a382ece0b3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:25.000288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd98fa-bfa1-4ba4-a27d-5a382ece0b3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:25.009364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd98fa-bfa1-4ba4-a27d-5a382ece0b3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:25.018297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd98fa-bfa1-4ba4-a27d-5a382ece0b3f service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:25.021112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 733/1449] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:27.299256 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b4e8c-d6ba-4fa4-9d6a-e2679657c5a4 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:27.299822 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b4e8c-d6ba-4fa4-9d6a-e2679657c5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:27.299822 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b4e8c-d6ba-4fa4-9d6a-e2679657c5a4 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:27.301214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:27.306731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a356c74-e03d-42e8-b7c5-85cca39f2003 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:27.307162 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a356c74-e03d-42e8-b7c5-85cca39f2003 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:27.307295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a356c74-e03d-42e8-b7c5-85cca39f2003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:27.327274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-7a356c74-e03d-42e8-b7c5-85cca39f2003 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:27.328465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7a356c74-e03d-42e8-b7c5-85cca39f2003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:27.336305 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:27.377192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b4e8c-d6ba-4fa4-9d6a-e2679657c5a4 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:27.388108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b4e8c-d6ba-4fa4-9d6a-e2679657c5a4 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:27.391632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a874460-2afa-4063-98f6-b69b04eeba70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:27.392265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a874460-2afa-4063-98f6-b69b04eeba70 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:27.392548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a874460-2afa-4063-98f6-b69b04eeba70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:27.392825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a874460-2afa-4063-98f6-b69b04eeba70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:27.402057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a874460-2afa-4063-98f6-b69b04eeba70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:27.404299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a874460-2afa-4063-98f6-b69b04eeba70 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:27.414044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 718/1452] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:29.724753 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837e115-42bc-4d09-8387-aed301e2b87c None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:29.725360 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837e115-42bc-4d09-8387-aed301e2b87c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:29.725360 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837e115-42bc-4d09-8387-aed301e2b87c None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:29.726734 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:29.732253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90200c8d-fecc-493f-b350-cb7455db9875 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:29.732673 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90200c8d-fecc-493f-b350-cb7455db9875 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:29.732806 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90200c8d-fecc-493f-b350-cb7455db9875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:29.750860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-90200c8d-fecc-493f-b350-cb7455db9875 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:29.752051 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-90200c8d-fecc-493f-b350-cb7455db9875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:29.759830 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:29.798177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837e115-42bc-4d09-8387-aed301e2b87c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:29.808754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837e115-42bc-4d09-8387-aed301e2b87c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:29.812845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2347f1-cea8-489d-af8f-c5da9521b1fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:29.813507 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2347f1-cea8-489d-af8f-c5da9521b1fb service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:29.813789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2347f1-cea8-489d-af8f-c5da9521b1fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:29.814060 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2347f1-cea8-489d-af8f-c5da9521b1fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:29.823461 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2347f1-cea8-489d-af8f-c5da9521b1fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:29.825788 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2347f1-cea8-489d-af8f-c5da9521b1fb service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:29.836090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 736/1455] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:32.050855 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53a58b-7751-4a1e-a763-4f5cc4275d28 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:32.051437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53a58b-7751-4a1e-a763-4f5cc4275d28 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:32.051437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a53a58b-7751-4a1e-a763-4f5cc4275d28 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:32.052767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:32.058083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327d9a7-aa27-403e-85a5-267fb63e5259 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:32.058603 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327d9a7-aa27-403e-85a5-267fb63e5259 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:32.058707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8327d9a7-aa27-403e-85a5-267fb63e5259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:32.065253 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=70 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:15:32.078054 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-8327d9a7-aa27-403e-85a5-267fb63e5259 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:32.079254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-8327d9a7-aa27-403e-85a5-267fb63e5259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:32.087119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:32.125347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53a58b-7751-4a1e-a763-4f5cc4275d28 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:32.130716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=64 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:15:32.136635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a53a58b-7751-4a1e-a763-4f5cc4275d28 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:32.141806 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895ec6b-68c1-4db3-bdee-08883b560150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:32.143661 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895ec6b-68c1-4db3-bdee-08883b560150 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:32.143661 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895ec6b-68c1-4db3-bdee-08883b560150 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:32.143661 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895ec6b-68c1-4db3-bdee-08883b560150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:32.152408 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895ec6b-68c1-4db3-bdee-08883b560150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:32.154675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895ec6b-68c1-4db3-bdee-08883b560150 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:32.163857 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 721/1458] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:34.448669 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac181c40-470b-437a-bbda-7207451b4b49 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:34.449269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac181c40-470b-437a-bbda-7207451b4b49 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:34.449269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac181c40-470b-437a-bbda-7207451b4b49 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:34.451398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:15:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:34.456653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7155373-f99e-48ee-8078-78ca1c0eafee None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:34.457100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7155373-f99e-48ee-8078-78ca1c0eafee None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:34.457238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7155373-f99e-48ee-8078-78ca1c0eafee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:34.474778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b7155373-f99e-48ee-8078-78ca1c0eafee None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:34.475965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b7155373-f99e-48ee-8078-78ca1c0eafee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:34.484240 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:34.521904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac181c40-470b-437a-bbda-7207451b4b49 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:34.532759 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac181c40-470b-437a-bbda-7207451b4b49 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:34.536255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecc33e-8841-4175-a7b5-592acab814c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:34.536902 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecc33e-8841-4175-a7b5-592acab814c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:34.537210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecc33e-8841-4175-a7b5-592acab814c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:34.537484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecc33e-8841-4175-a7b5-592acab814c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:34.547360 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ecc33e-8841-4175-a7b5-592acab814c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:34.550439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ecc33e-8841-4175-a7b5-592acab814c3 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:34.565583 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 739/1461] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:15:36.789977 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2190832-1fd3-4a47-ac1f-7244d820b462 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:36.790580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2190832-1fd3-4a47-ac1f-7244d820b462 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:36.790580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2190832-1fd3-4a47-ac1f-7244d820b462 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:36.791805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:36.797156 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f547d2-72ad-4375-a13b-40cfc36e20c9 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:36.797590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f547d2-72ad-4375-a13b-40cfc36e20c9 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:36.797702 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f547d2-72ad-4375-a13b-40cfc36e20c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:36.815644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-43f547d2-72ad-4375-a13b-40cfc36e20c9 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:36.816841 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-43f547d2-72ad-4375-a13b-40cfc36e20c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:36.825647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:36.863243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2190832-1fd3-4a47-ac1f-7244d820b462 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:36.875067 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2190832-1fd3-4a47-ac1f-7244d820b462 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:36.878641 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee6424-b864-4c02-95dc-cc04ea1f3d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:36.879282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee6424-b864-4c02-95dc-cc04ea1f3d9d service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:36.879565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee6424-b864-4c02-95dc-cc04ea1f3d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:36.879840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee6424-b864-4c02-95dc-cc04ea1f3d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:36.890441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee6424-b864-4c02-95dc-cc04ea1f3d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:36.891861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee6424-b864-4c02-95dc-cc04ea1f3d9d service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:36.901355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 724/1464] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:39.117967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1d2f3-4892-4cff-b604-d5b30c7c58aa None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:39.118620 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1d2f3-4892-4cff-b604-d5b30c7c58aa None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:39.118620 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1d2f3-4892-4cff-b604-d5b30c7c58aa None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:39.119929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:39.127556 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdd42f-9e01-4ebd-a5a9-62d440bdba3c None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:39.131952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdd42f-9e01-4ebd-a5a9-62d440bdba3c None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:39.131952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdd42f-9e01-4ebd-a5a9-62d440bdba3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:39.147309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-6ffdd42f-9e01-4ebd-a5a9-62d440bdba3c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:39.148495 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-6ffdd42f-9e01-4ebd-a5a9-62d440bdba3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:39.158106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:15:39.197248 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1d2f3-4892-4cff-b604-d5b30c7c58aa None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:39.208272 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1d2f3-4892-4cff-b604-d5b30c7c58aa None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:39.211798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df218651-320c-48e7-95b4-854a03316cdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:39.212435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df218651-320c-48e7-95b4-854a03316cdb service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:39.212716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df218651-320c-48e7-95b4-854a03316cdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:39.213025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df218651-320c-48e7-95b4-854a03316cdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:39.222622 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df218651-320c-48e7-95b4-854a03316cdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:39.224097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df218651-320c-48e7-95b4-854a03316cdb service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:39.235193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 742/1467] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:41.396405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71d84c-c30f-4e1d-bc24-33097de60a49 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:41.396990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71d84c-c30f-4e1d-bc24-33097de60a49 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:41.396990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71d84c-c30f-4e1d-bc24-33097de60a49 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:41.398351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:15:41.404476 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b1d70-bec2-4bc2-9c2f-e77c3a7bcd1c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:41.404942 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b1d70-bec2-4bc2-9c2f-e77c3a7bcd1c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:41.405124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b1d70-bec2-4bc2-9c2f-e77c3a7bcd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:41.422800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-836b1d70-bec2-4bc2-9c2f-e77c3a7bcd1c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:15:41.423979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-836b1d70-bec2-4bc2-9c2f-e77c3a7bcd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:15:41.431986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:15:41.470522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71d84c-c30f-4e1d-bc24-33097de60a49 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:15:41.481050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71d84c-c30f-4e1d-bc24-33097de60a49 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:15:41.484519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d1b74-dabf-4ee0-9f2c-efa4b9fee463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:15:41.485178 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d1b74-dabf-4ee0-9f2c-efa4b9fee463 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:15:41.485464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d1b74-dabf-4ee0-9f2c-efa4b9fee463 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:15:41.485742 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d1b74-dabf-4ee0-9f2c-efa4b9fee463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:15:41.494820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d1b74-dabf-4ee0-9f2c-efa4b9fee463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:15:41.496231 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d1b74-dabf-4ee0-9f2c-efa4b9fee463 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:15:41.505548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 727/1470] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:15:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:15:51.417638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=20 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:15:51.501424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=23 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:16:02.205108 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966c023-8dc3-41c1-b2fd-743fd22c6876 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:02.205717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966c023-8dc3-41c1-b2fd-743fd22c6876 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:02.205717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966c023-8dc3-41c1-b2fd-743fd22c6876 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:02.207140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 744/1471] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:16:02.212793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188361b5-835b-44ff-813e-bc802133dc80 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:02.213275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188361b5-835b-44ff-813e-bc802133dc80 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:02.213403 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188361b5-835b-44ff-813e-bc802133dc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:02.232405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-188361b5-835b-44ff-813e-bc802133dc80 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:16:02.233624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-188361b5-835b-44ff-813e-bc802133dc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:16:02.242349 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 728/1472] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:16:02.291475 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75fa62-023c-479a-8eea-e939c2553f1c None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:02.291766 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75fa62-023c-479a-8eea-e939c2553f1c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:02.292042 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75fa62-023c-479a-8eea-e939c2553f1c None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:02.293523 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 745/1473] 10.222.0.60 () {54 vars in 962 bytes} [Wed Jun 15 10:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:16:02.298388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565907d-b1d9-4efa-a624-6d31a51998dc None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:02.298742 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565907d-b1d9-4efa-a624-6d31a51998dc None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:02.299025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565907d-b1d9-4efa-a624-6d31a51998dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:02.318221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-4565907d-b1d9-4efa-a624-6d31a51998dc None None] MFA Rules not processed for user `7fbf33189aa94f4f973d558d615a23e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:16:02.319431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4565907d-b1d9-4efa-a624-6d31a51998dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:16:02.326647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 729/1474] 10.222.0.60 () {58 vars in 1059 bytes} [Wed Jun 15 10:16:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:16:02.334130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75fa62-023c-479a-8eea-e939c2553f1c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:16:02.345512 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75fa62-023c-479a-8eea-e939c2553f1c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:16:02.348924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5d87c7-9fb1-44a4-8779-4c09ed57b97c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fbf33189aa94f4f973d558d615a23e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:16:02.349575 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d87c7-9fb1-44a4-8779-4c09ed57b97c service glance] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:02.349856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d87c7-9fb1-44a4-8779-4c09ed57b97c service glance] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:02.350128 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d87c7-9fb1-44a4-8779-4c09ed57b97c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:02.359019 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d87c7-9fb1-44a4-8779-4c09ed57b97c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:16:02.360428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5d87c7-9fb1-44a4-8779-4c09ed57b97c service glance] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:16:02.369642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 746/1475] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:16:04.648987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa1b86-5c67-4bdb-89c2-797c4d2d92cb None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:04.649552 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa1b86-5c67-4bdb-89c2-797c4d2d92cb None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:04.649745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa1b86-5c67-4bdb-89c2-797c4d2d92cb None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:04.651142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 730/1476] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:16:04.656743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e822ef-1744-4b72-95e8-447b102bb5e0 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:04.657295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e822ef-1744-4b72-95e8-447b102bb5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:04.657295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e822ef-1744-4b72-95e8-447b102bb5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:04.675666 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-40e822ef-1744-4b72-95e8-447b102bb5e0 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:16:04.676874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-40e822ef-1744-4b72-95e8-447b102bb5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:16:04.685697 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 747/1477] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:16:04.709619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa1b86-5c67-4bdb-89c2-797c4d2d92cb None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:16:04.722484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa1b86-5c67-4bdb-89c2-797c4d2d92cb None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:16:04.726818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69367e-117a-46ac-b703-8f2ae4baf549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:16:04.727465 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69367e-117a-46ac-b703-8f2ae4baf549 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:04.727745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69367e-117a-46ac-b703-8f2ae4baf549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:04.728015 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69367e-117a-46ac-b703-8f2ae4baf549 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:04.728826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69367e-117a-46ac-b703-8f2ae4baf549 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:16:04.731668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: WARNING py.warnings [None req-ab69367e-117a-46ac-b703-8f2ae4baf549 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:16:04.731668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: warnings.warn(msg) Jun 15 10:16:04.731668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: Jun 15 10:16:04.732815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69367e-117a-46ac-b703-8f2ae4baf549 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:16:04.753795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 731/1478] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jun 15 10:16:04 2022] GET /identity/v3/projects => generated 2553 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:16:06.948856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6e5df-216d-4b06-a109-ee3ac4fbf152 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:06.949441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6e5df-216d-4b06-a109-ee3ac4fbf152 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:06.949441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6e5df-216d-4b06-a109-ee3ac4fbf152 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:06.950948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 748/1479] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:16:06.956344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578f29d-76be-4e93-b302-58d536ef38d4 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:06.956788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578f29d-76be-4e93-b302-58d536ef38d4 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:06.956904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578f29d-76be-4e93-b302-58d536ef38d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:06.976122 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b578f29d-76be-4e93-b302-58d536ef38d4 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:16:06.977438 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b578f29d-76be-4e93-b302-58d536ef38d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:16:06.986404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 732/1480] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:16:07.030217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6e5df-216d-4b06-a109-ee3ac4fbf152 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:16:07.045196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6e5df-216d-4b06-a109-ee3ac4fbf152 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:16:07.051584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426a75d-5e7b-4253-a17e-3ffdac1130d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:16:07.052268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426a75d-5e7b-4253-a17e-3ffdac1130d3 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:07.052554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426a75d-5e7b-4253-a17e-3ffdac1130d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:07.052832 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426a75d-5e7b-4253-a17e-3ffdac1130d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:07.062343 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426a75d-5e7b-4253-a17e-3ffdac1130d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:16:07.063939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426a75d-5e7b-4253-a17e-3ffdac1130d3 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:16:07.074013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 749/1481] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:16:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:16:09.440977 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec0da0-3f1b-4ff2-ae29-bafc837485ad None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:09.441522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec0da0-3f1b-4ff2-ae29-bafc837485ad None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:09.441642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec0da0-3f1b-4ff2-ae29-bafc837485ad None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:09.443202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 733/1482] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:16:09.448459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99e42f-2689-435a-aabd-68448a52b33b None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:09.450045 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99e42f-2689-435a-aabd-68448a52b33b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:09.450334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c99e42f-2689-435a-aabd-68448a52b33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:09.468124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-2c99e42f-2689-435a-aabd-68448a52b33b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:16:09.469342 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2c99e42f-2689-435a-aabd-68448a52b33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:16:09.477105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 750/1483] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:16:09.515165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec0da0-3f1b-4ff2-ae29-bafc837485ad None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:16:09.527551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec0da0-3f1b-4ff2-ae29-bafc837485ad None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:16:09.531065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17ab26-155b-420a-830a-07fe0bc46b06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:16:09.531702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17ab26-155b-420a-830a-07fe0bc46b06 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:09.531980 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17ab26-155b-420a-830a-07fe0bc46b06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:09.532254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17ab26-155b-420a-830a-07fe0bc46b06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:09.541766 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17ab26-155b-420a-830a-07fe0bc46b06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:16:09.543642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17ab26-155b-420a-830a-07fe0bc46b06 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:16:09.553236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 734/1484] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:16:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:16:11.780913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606fcf7-1488-4bb3-be2a-53cbf17b7f8e None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:11.781504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606fcf7-1488-4bb3-be2a-53cbf17b7f8e None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:11.781504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606fcf7-1488-4bb3-be2a-53cbf17b7f8e None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:11.782908 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 751/1485] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:16:11.788509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412c155-9c7c-4074-a6c3-69754286e110 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:11.789024 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412c155-9c7c-4074-a6c3-69754286e110 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:11.789132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412c155-9c7c-4074-a6c3-69754286e110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:11.806771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-f412c155-9c7c-4074-a6c3-69754286e110 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:16:11.808489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f412c155-9c7c-4074-a6c3-69754286e110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:16:11.816766 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 735/1486] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:16:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:16:11.855585 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606fcf7-1488-4bb3-be2a-53cbf17b7f8e None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:16:11.866208 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606fcf7-1488-4bb3-be2a-53cbf17b7f8e None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:16:11.869694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4ee75-454e-41b7-8769-26f57d6d0951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:16:11.870807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4ee75-454e-41b7-8769-26f57d6d0951 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:11.871103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4ee75-454e-41b7-8769-26f57d6d0951 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:11.871382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4ee75-454e-41b7-8769-26f57d6d0951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:11.880939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4ee75-454e-41b7-8769-26f57d6d0951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:16:11.882403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4ee75-454e-41b7-8769-26f57d6d0951 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:16:11.893430 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 752/1487] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:16:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:16:14.233580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c7d52-af9c-4faf-af57-30f742c43f6e None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:14.234166 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c7d52-af9c-4faf-af57-30f742c43f6e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:14.234166 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c7d52-af9c-4faf-af57-30f742c43f6e None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:14.235539 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 736/1488] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:16:14.240710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b808a-7c12-481f-be67-33ad343bf52d None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:14.241158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b808a-7c12-481f-be67-33ad343bf52d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:14.241313 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b808a-7c12-481f-be67-33ad343bf52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:14.259240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-5f4b808a-7c12-481f-be67-33ad343bf52d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:16:14.260419 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5f4b808a-7c12-481f-be67-33ad343bf52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:16:14.268282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 753/1489] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:16:14.311065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c7d52-af9c-4faf-af57-30f742c43f6e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:16:14.321215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c7d52-af9c-4faf-af57-30f742c43f6e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:16:14.324798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa404f-d343-4ad5-8265-11566d2f0e73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:16:14.325461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa404f-d343-4ad5-8265-11566d2f0e73 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:14.325774 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa404f-d343-4ad5-8265-11566d2f0e73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:14.326049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa404f-d343-4ad5-8265-11566d2f0e73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:14.334834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa404f-d343-4ad5-8265-11566d2f0e73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:16:14.336247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa404f-d343-4ad5-8265-11566d2f0e73 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:16:14.345697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 737/1490] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:16:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:16:16.881348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad478a-fc46-4e86-b4db-b57f6f1b116e None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:16.881968 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad478a-fc46-4e86-b4db-b57f6f1b116e None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:16.881968 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad478a-fc46-4e86-b4db-b57f6f1b116e None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:16.883309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 754/1491] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jun 15 10:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:16:16.889407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137e610-4df1-468b-96e0-eb7d29d6c46d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:16.889838 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137e610-4df1-468b-96e0-eb7d29d6c46d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:16.889957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137e610-4df1-468b-96e0-eb7d29d6c46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:16.907970 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-8137e610-4df1-468b-96e0-eb7d29d6c46d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:16:16.909191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-8137e610-4df1-468b-96e0-eb7d29d6c46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:16:16.916937 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 738/1492] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jun 15 10:16:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:16:16.957257 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad478a-fc46-4e86-b4db-b57f6f1b116e None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:16:16.969344 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad478a-fc46-4e86-b4db-b57f6f1b116e None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:16:16.973005 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacdc8e-eac7-49e9-9d38-2c65d35e3610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:16:16.973672 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacdc8e-eac7-49e9-9d38-2c65d35e3610 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:16:16.973962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacdc8e-eac7-49e9-9d38-2c65d35e3610 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:16:16.974243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacdc8e-eac7-49e9-9d38-2c65d35e3610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:16:16.984017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacdc8e-eac7-49e9-9d38-2c65d35e3610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:16:16.985530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacdc8e-eac7-49e9-9d38-2c65d35e3610 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:16:16.995925 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 755/1493] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:16:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:16:26.903363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=33 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:16:26.990964 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=33 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:54.511243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d678c-d566-4a5f-88b1-43c98251ba4e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:54.511905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d678c-d566-4a5f-88b1-43c98251ba4e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:54.511905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d678c-d566-4a5f-88b1-43c98251ba4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:54.530404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b75d678c-d566-4a5f-88b1-43c98251ba4e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:54.531606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b75d678c-d566-4a5f-88b1-43c98251ba4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:54.539903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 739/1494] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:17:54.549584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e434464-f2d5-483f-a414-b6ef2fe2d53a None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:54.550074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e434464-f2d5-483f-a414-b6ef2fe2d53a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:54.550180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e434464-f2d5-483f-a414-b6ef2fe2d53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:54.568411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-7e434464-f2d5-483f-a414-b6ef2fe2d53a None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:54.569636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7e434464-f2d5-483f-a414-b6ef2fe2d53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:54.578285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 756/1495] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:17:54.583007 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d678c-d566-4a5f-88b1-43c98251ba4e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:54.593684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d678c-d566-4a5f-88b1-43c98251ba4e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:54.598152 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd8705-c49b-4b6e-a403-20cee4b2d933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:54.598810 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd8705-c49b-4b6e-a403-20cee4b2d933 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:54.599091 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd8705-c49b-4b6e-a403-20cee4b2d933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:54.599363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd8705-c49b-4b6e-a403-20cee4b2d933 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:54.600125 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd8705-c49b-4b6e-a403-20cee4b2d933 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:17:54.600431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd8705-c49b-4b6e-a403-20cee4b2d933 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:54.601612 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: WARNING py.warnings [None req-8edd8705-c49b-4b6e-a403-20cee4b2d933 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:17:54.601612 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: warnings.warn(msg) Jun 15 10:17:54.601612 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: Jun 15 10:17:54.602497 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd8705-c49b-4b6e-a403-20cee4b2d933 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:54.614275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 740/1496] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:17:54.622789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e434464-f2d5-483f-a414-b6ef2fe2d53a None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:54.631866 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e434464-f2d5-483f-a414-b6ef2fe2d53a None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:54.636914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc865a00-1fad-4471-9f36-59c762926c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:54.637588 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc865a00-1fad-4471-9f36-59c762926c24 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:54.637872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc865a00-1fad-4471-9f36-59c762926c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:54.638149 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc865a00-1fad-4471-9f36-59c762926c24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:54.638832 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc865a00-1fad-4471-9f36-59c762926c24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:54.640585 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: WARNING py.warnings [None req-bc865a00-1fad-4471-9f36-59c762926c24 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:17:54.640585 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: warnings.warn(msg) Jun 15 10:17:54.640585 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: Jun 15 10:17:54.641799 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc865a00-1fad-4471-9f36-59c762926c24 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:54.783912 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 757/1497] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:17:54 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:17:54.788228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd8705-c49b-4b6e-a403-20cee4b2d933 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:54.798652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd8705-c49b-4b6e-a403-20cee4b2d933 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:54.858384 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a62e41-ff2d-472a-b752-fd2921b22e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:54.863996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a62e41-ff2d-472a-b752-fd2921b22e00 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:54.863996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a62e41-ff2d-472a-b752-fd2921b22e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:54.863996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a62e41-ff2d-472a-b752-fd2921b22e00 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:54.863996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a62e41-ff2d-472a-b752-fd2921b22e00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:54.863996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: WARNING py.warnings [None req-a3a62e41-ff2d-472a-b752-fd2921b22e00 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:17:54.863996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: warnings.warn(msg) Jun 15 10:17:54.863996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: Jun 15 10:17:54.864605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a62e41-ff2d-472a-b752-fd2921b22e00 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:54.965938 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 741/1498] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:17:54 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:17:54.970197 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc865a00-1fad-4471-9f36-59c762926c24 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:54.981480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc865a00-1fad-4471-9f36-59c762926c24 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:54.986200 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e075b-bebe-4d7d-96d8-b348e0bfe383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:54.986847 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e075b-bebe-4d7d-96d8-b348e0bfe383 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:54.987166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e075b-bebe-4d7d-96d8-b348e0bfe383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:54.987450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e075b-bebe-4d7d-96d8-b348e0bfe383 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:54.988190 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e075b-bebe-4d7d-96d8-b348e0bfe383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:54.989327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: WARNING py.warnings [None req-e90e075b-bebe-4d7d-96d8-b348e0bfe383 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:17:54.989327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: warnings.warn(msg) Jun 15 10:17:54.989327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: Jun 15 10:17:54.990213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e075b-bebe-4d7d-96d8-b348e0bfe383 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:55.003775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 758/1499] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:17:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:17:55.007990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a62e41-ff2d-472a-b752-fd2921b22e00 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:55.018212 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a62e41-ff2d-472a-b752-fd2921b22e00 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:55.022849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a67d2-09e2-431c-b037-ebf618910d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:55.023518 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a67d2-09e2-431c-b037-ebf618910d49 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.023800 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a67d2-09e2-431c-b037-ebf618910d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.024078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a67d2-09e2-431c-b037-ebf618910d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.024827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a67d2-09e2-431c-b037-ebf618910d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:55.026357 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: WARNING py.warnings [None req-9f1a67d2-09e2-431c-b037-ebf618910d49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:17:55.026357 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: warnings.warn(msg) Jun 15 10:17:55.026357 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: Jun 15 10:17:55.027240 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a67d2-09e2-431c-b037-ebf618910d49 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:55.042093 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 742/1500] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:17:55.046300 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e075b-bebe-4d7d-96d8-b348e0bfe383 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:55.056464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e075b-bebe-4d7d-96d8-b348e0bfe383 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:55.061174 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8029b4-ad06-4048-b3af-43e64922d196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:55.062063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8029b4-ad06-4048-b3af-43e64922d196 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.062350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8029b4-ad06-4048-b3af-43e64922d196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.062635 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8029b4-ad06-4048-b3af-43e64922d196 admin admin] PATH_INFO: `/v3/projects/22541a156c7e4232aab7643df8e5c214/users/3bb4c60a8f6b46d7ab78c78ff39070d8/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.078583 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8029b4-ad06-4048-b3af-43e64922d196 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22541a156c7e4232aab7643df8e5c214, user_id=3bb4c60a8f6b46d7ab78c78ff39070d8, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:55.079643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: WARNING py.warnings [None req-bc8029b4-ad06-4048-b3af-43e64922d196 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:17:55.079643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: warnings.warn(msg) Jun 15 10:17:55.079643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: Jun 15 10:17:55.081621 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8029b4-ad06-4048-b3af-43e64922d196 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:55.177474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 759/1501] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:17:55 2022] PUT /identity/v3/projects/22541a156c7e4232aab7643df8e5c214/users/3bb4c60a8f6b46d7ab78c78ff39070d8/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:17:55.189137 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7910df-6128-46d9-99a6-97a8e45b08ac None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.189476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7910df-6128-46d9-99a6-97a8e45b08ac None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.189759 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7910df-6128-46d9-99a6-97a8e45b08ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.218307 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-4e7910df-6128-46d9-99a6-97a8e45b08ac None None] MFA Rules not processed for user `3bb4c60a8f6b46d7ab78c78ff39070d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:55.219538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4e7910df-6128-46d9-99a6-97a8e45b08ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:55.284831 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 743/1502] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:17:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:17:55.298630 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04233e3-4962-4bb0-ba2c-09fad510a152 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.298981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04233e3-4962-4bb0-ba2c-09fad510a152 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.299368 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04233e3-4962-4bb0-ba2c-09fad510a152 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.301394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 760/1503] 10.222.0.60 () {54 vars in 963 bytes} [Wed Jun 15 10:17:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:55.306671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873daea9-3605-43d3-aa39-60ea4f843ab1 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.306671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873daea9-3605-43d3-aa39-60ea4f843ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.307296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873daea9-3605-43d3-aa39-60ea4f843ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.326554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-873daea9-3605-43d3-aa39-60ea4f843ab1 None None] MFA Rules not processed for user `fa51162a69d44127bd34df2f4653dc15`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:55.327759 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-873daea9-3605-43d3-aa39-60ea4f843ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:55.371624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:55.379786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 744/1504] 10.222.0.60 () {58 vars in 1060 bytes} [Wed Jun 15 10:17:55 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:17:55.388111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04233e3-4962-4bb0-ba2c-09fad510a152 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:55.398799 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04233e3-4962-4bb0-ba2c-09fad510a152 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:55.403178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ce5e3-cc7a-4485-8eb7-49d4c7039f88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:55.403850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ce5e3-cc7a-4485-8eb7-49d4c7039f88 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.404132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ce5e3-cc7a-4485-8eb7-49d4c7039f88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.404410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ce5e3-cc7a-4485-8eb7-49d4c7039f88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.413530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ce5e3-cc7a-4485-8eb7-49d4c7039f88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:55.415148 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ce5e3-cc7a-4485-8eb7-49d4c7039f88 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:55.424191 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 761/1505] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:17:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:17:55.468668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797b7d5-4caf-48c8-92a1-46cf9e43fe0a None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.468982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797b7d5-4caf-48c8-92a1-46cf9e43fe0a None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.469273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797b7d5-4caf-48c8-92a1-46cf9e43fe0a None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.470572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 745/1506] 10.222.0.60 () {52 vars in 829 bytes} [Wed Jun 15 10:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:17:55.474332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ce5e3-cc7a-4485-8eb7-49d4c7039f88 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:55.540609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ce5e3-cc7a-4485-8eb7-49d4c7039f88 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:55.545199 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4c4c2-3e39-4595-a83d-0531dc067bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:55.546082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4c4c2-3e39-4595-a83d-0531dc067bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.546367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4c4c2-3e39-4595-a83d-0531dc067bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.546646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4c4c2-3e39-4595-a83d-0531dc067bca admin admin] PATH_INFO: `/v3/users/3bb4c60a8f6b46d7ab78c78ff39070d8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.547850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4c4c2-3e39-4595-a83d-0531dc067bca admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bb4c60a8f6b46d7ab78c78ff39070d8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:55.548987 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: WARNING py.warnings [None req-0bf4c4c2-3e39-4595-a83d-0531dc067bca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:17:55.548987 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: warnings.warn(msg) Jun 15 10:17:55.548987 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: Jun 15 10:17:55.550152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4c4c2-3e39-4595-a83d-0531dc067bca admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:55.767508 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0bf4c4c2-3e39-4595-a83d-0531dc067bca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bb4c60a8f6b46d7ab78c78ff39070d8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:17:55.787024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0bf4c4c2-3e39-4595-a83d-0531dc067bca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bb4c60a8f6b46d7ab78c78ff39070d8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:17:55.856453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0bf4c4c2-3e39-4595-a83d-0531dc067bca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bb4c60a8f6b46d7ab78c78ff39070d8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:17:55.866659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:17:55.872007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0bf4c4c2-3e39-4595-a83d-0531dc067bca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bb4c60a8f6b46d7ab78c78ff39070d8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:17:55.872929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0bf4c4c2-3e39-4595-a83d-0531dc067bca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bb4c60a8f6b46d7ab78c78ff39070d8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:17:55.880167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 762/1507] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:17:55 2022] DELETE /identity/v3/users/3bb4c60a8f6b46d7ab78c78ff39070d8 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:17:55.885360 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242d46b-d5a4-402e-a1a9-52c650ce8fcf None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:55.886820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242d46b-d5a4-402e-a1a9-52c650ce8fcf None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:55.886820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242d46b-d5a4-402e-a1a9-52c650ce8fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:55.904215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-1242d46b-d5a4-402e-a1a9-52c650ce8fcf None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:17:55.905445 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1242d46b-d5a4-402e-a1a9-52c650ce8fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:55.967659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 746/1508] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:17:55.977453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4c4c2-3e39-4595-a83d-0531dc067bca admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:55.978942 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0bf4c4c2-3e39-4595-a83d-0531dc067bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:56.036028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4c4c2-3e39-4595-a83d-0531dc067bca admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:56.039784 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90186901-220c-4bdb-b534-2ae9a81a188f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:56.040473 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90186901-220c-4bdb-b534-2ae9a81a188f service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:56.040845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90186901-220c-4bdb-b534-2ae9a81a188f service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:56.041157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90186901-220c-4bdb-b534-2ae9a81a188f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:56.050625 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90186901-220c-4bdb-b534-2ae9a81a188f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:56.052009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90186901-220c-4bdb-b534-2ae9a81a188f service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:56.061784 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 763/1509] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:17:56.539747 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242d46b-d5a4-402e-a1a9-52c650ce8fcf None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:17:56.543052 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1242d46b-d5a4-402e-a1a9-52c650ce8fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:17:56.554300 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242d46b-d5a4-402e-a1a9-52c650ce8fcf None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:17:56.559202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9002c4db-34a0-4c8c-91d2-433a25744cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:17:56.564761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002c4db-34a0-4c8c-91d2-433a25744cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:17:56.565077 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002c4db-34a0-4c8c-91d2-433a25744cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:17:56.565361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002c4db-34a0-4c8c-91d2-433a25744cfb admin admin] PATH_INFO: `/v3/projects/22541a156c7e4232aab7643df8e5c214` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:17:56.566639 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002c4db-34a0-4c8c-91d2-433a25744cfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=22541a156c7e4232aab7643df8e5c214)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:17:56.567674 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: WARNING py.warnings [None req-9002c4db-34a0-4c8c-91d2-433a25744cfb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:17:56.567674 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: warnings.warn(msg) Jun 15 10:17:56.567674 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: Jun 15 10:17:56.569181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9002c4db-34a0-4c8c-91d2-433a25744cfb admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:17:56.691713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9002c4db-34a0-4c8c-91d2-433a25744cfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22541a156c7e4232aab7643df8e5c214', 'request_id': 'req-9002c4db-34a0-4c8c-91d2-433a25744cfb'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:17:56.706337 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9002c4db-34a0-4c8c-91d2-433a25744cfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22541a156c7e4232aab7643df8e5c214', 'request_id': 'req-9002c4db-34a0-4c8c-91d2-433a25744cfb'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:17:56.799502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9002c4db-34a0-4c8c-91d2-433a25744cfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22541a156c7e4232aab7643df8e5c214', 'request_id': 'req-9002c4db-34a0-4c8c-91d2-433a25744cfb'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:17:56.811842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9002c4db-34a0-4c8c-91d2-433a25744cfb admin admin] The token cache is being invalidate because project 22541a156c7e4232aab7643df8e5c214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:17:56.812131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9002c4db-34a0-4c8c-91d2-433a25744cfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:17:56.812932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9002c4db-34a0-4c8c-91d2-433a25744cfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:17:56.814700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 747/1510] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:17:56 2022] DELETE /identity/v3/projects/22541a156c7e4232aab7643df8e5c214 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:18:06.057492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=29 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:18:06.808496 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:40.809565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a09d6c-811f-4a0f-aed9-5685cd3af816 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:40.810502 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a09d6c-811f-4a0f-aed9-5685cd3af816 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:40.810502 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a09d6c-811f-4a0f-aed9-5685cd3af816 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:40.812079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:19:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:40.818319 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e299f-a0ac-4d28-ae70-6cb19719b0c3 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:40.818925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e299f-a0ac-4d28-ae70-6cb19719b0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:40.819073 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e299f-a0ac-4d28-ae70-6cb19719b0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:40.839202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-737e299f-a0ac-4d28-ae70-6cb19719b0c3 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:40.840439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-737e299f-a0ac-4d28-ae70-6cb19719b0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:40.906736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:19:41.093987 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a09d6c-811f-4a0f-aed9-5685cd3af816 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:41.095866 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-41a09d6c-811f-4a0f-aed9-5685cd3af816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:41.194057 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a09d6c-811f-4a0f-aed9-5685cd3af816 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:41.201466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f6f8e-3535-4839-8872-d77134d2dbc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:41.202145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f6f8e-3535-4839-8872-d77134d2dbc9 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:41.202592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f6f8e-3535-4839-8872-d77134d2dbc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:41.202910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f6f8e-3535-4839-8872-d77134d2dbc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:41.212802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f6f8e-3535-4839-8872-d77134d2dbc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:41.214582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f6f8e-3535-4839-8872-d77134d2dbc9 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:41.225862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 765/1513] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:19:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:19:42.224987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1b098-1eb1-40d6-a5bf-a900e5d11626 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:42.225594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1b098-1eb1-40d6-a5bf-a900e5d11626 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:42.226004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1b098-1eb1-40d6-a5bf-a900e5d11626 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:42.229826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:19:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:42.235603 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18a243-3eba-4613-98d1-6086f5a934cd None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:42.236155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18a243-3eba-4613-98d1-6086f5a934cd None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:42.236155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18a243-3eba-4613-98d1-6086f5a934cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:42.257426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-6f18a243-3eba-4613-98d1-6086f5a934cd None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:42.259694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6f18a243-3eba-4613-98d1-6086f5a934cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:42.268686 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:19:42.426078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1b098-1eb1-40d6-a5bf-a900e5d11626 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:42.441887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1b098-1eb1-40d6-a5bf-a900e5d11626 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:42.449569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbb4bf-0ea6-41d0-98d5-53a0a8ea325c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:42.450277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbb4bf-0ea6-41d0-98d5-53a0a8ea325c service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:42.450634 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbb4bf-0ea6-41d0-98d5-53a0a8ea325c service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:42.450957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbb4bf-0ea6-41d0-98d5-53a0a8ea325c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:42.460425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbb4bf-0ea6-41d0-98d5-53a0a8ea325c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:42.463322 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbb4bf-0ea6-41d0-98d5-53a0a8ea325c service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:42.475102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 750/1516] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:19:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:19:42.633376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2a664-3898-4125-af47-3b096269b47e None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:42.633750 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2a664-3898-4125-af47-3b096269b47e None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:42.634101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2a664-3898-4125-af47-3b096269b47e None None] PATH_INFO: `/v3` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:42.635599 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 767/1517] 10.222.0.58 () {54 vars in 925 bytes} [Wed Jun 15 10:19:42 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:19:42.645570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35defe-a57b-4c81-9a3b-2d54177c20ae None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:42.645570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35defe-a57b-4c81-9a3b-2d54177c20ae None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:42.645570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35defe-a57b-4c81-9a3b-2d54177c20ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:42.667690 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-0b35defe-a57b-4c81-9a3b-2d54177c20ae None None] MFA Rules not processed for user `12899a7a59b54e3199b277ff94e30d42`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:42.669573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-0b35defe-a57b-4c81-9a3b-2d54177c20ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:42.760004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 751/1518] 10.222.0.58 () {58 vars in 1015 bytes} [Wed Jun 15 10:19:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:19:52.130429 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f780df-84a1-4ce7-a688-7bcfbfb1b0aa None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:52.131128 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f780df-84a1-4ce7-a688-7bcfbfb1b0aa None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:52.131128 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f780df-84a1-4ce7-a688-7bcfbfb1b0aa None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:52.132659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 768/1519] 10.222.0.60 () {54 vars in 960 bytes} [Wed Jun 15 10:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:52.138591 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8877c-21f0-4c09-ac48-64b868ffe19e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:52.139089 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8877c-21f0-4c09-ac48-64b868ffe19e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:52.139192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8877c-21f0-4c09-ac48-64b868ffe19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:52.159528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-7bd8877c-21f0-4c09-ac48-64b868ffe19e None None] MFA Rules not processed for user `12899a7a59b54e3199b277ff94e30d42`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:52.161286 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-7bd8877c-21f0-4c09-ac48-64b868ffe19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:52.172983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 752/1520] 10.222.0.60 () {58 vars in 1057 bytes} [Wed Jun 15 10:19:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:19:52.180677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f780df-84a1-4ce7-a688-7bcfbfb1b0aa None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:52.196182 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f780df-84a1-4ce7-a688-7bcfbfb1b0aa None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:52.201940 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07be5df5-5393-4e1e-8397-ac86ef9999fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:52.202596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be5df5-5393-4e1e-8397-ac86ef9999fb service placement] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:52.202941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be5df5-5393-4e1e-8397-ac86ef9999fb service placement] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:52.203326 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07be5df5-5393-4e1e-8397-ac86ef9999fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:52.216063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be5df5-5393-4e1e-8397-ac86ef9999fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:52.218520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07be5df5-5393-4e1e-8397-ac86ef9999fb service placement] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:52.231056 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 769/1521] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:19:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:19:53.860386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d386eb-3127-4626-894c-29964ec19d2b None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:53.860903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d386eb-3127-4626-894c-29964ec19d2b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:53.860903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d386eb-3127-4626-894c-29964ec19d2b None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:53.862303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:53.868575 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648291c-c245-4746-9535-07b846d7a662 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:53.869099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648291c-c245-4746-9535-07b846d7a662 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:53.869274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648291c-c245-4746-9535-07b846d7a662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:53.888645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-7648291c-c245-4746-9535-07b846d7a662 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:53.890133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7648291c-c245-4746-9535-07b846d7a662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:53.898551 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:19:54.060644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d386eb-3127-4626-894c-29964ec19d2b None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:54.077084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d386eb-3127-4626-894c-29964ec19d2b None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:54.082636 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be5c8a-b6af-4be6-b19a-640f5802cc44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:54.083292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be5c8a-b6af-4be6-b19a-640f5802cc44 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:54.083577 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be5c8a-b6af-4be6-b19a-640f5802cc44 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:54.083857 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be5c8a-b6af-4be6-b19a-640f5802cc44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:54.092908 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be5c8a-b6af-4be6-b19a-640f5802cc44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:54.094578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be5c8a-b6af-4be6-b19a-640f5802cc44 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:54.105559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 754/1524] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:19:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:19:54.997196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb9cdc-ce3f-40ca-8ca5-7ba73f458330 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:54.997779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb9cdc-ce3f-40ca-8ca5-7ba73f458330 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:54.997779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb9cdc-ce3f-40ca-8ca5-7ba73f458330 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:54.999149 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 15 10:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:55.004713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269392d3-9811-4e65-a31f-ec9dc334dad0 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:55.005201 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269392d3-9811-4e65-a31f-ec9dc334dad0 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:55.005309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269392d3-9811-4e65-a31f-ec9dc334dad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:55.024597 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-269392d3-9811-4e65-a31f-ec9dc334dad0 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:55.025862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-269392d3-9811-4e65-a31f-ec9dc334dad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:55.033925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 15 10:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:19:55.215343 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb9cdc-ce3f-40ca-8ca5-7ba73f458330 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:55.233991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb9cdc-ce3f-40ca-8ca5-7ba73f458330 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:55.239927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d47d6-1325-481c-a935-2ba318db062d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:55.240593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d47d6-1325-481c-a935-2ba318db062d service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:55.240881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d47d6-1325-481c-a935-2ba318db062d service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:55.241199 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d47d6-1325-481c-a935-2ba318db062d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:55.250348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d47d6-1325-481c-a935-2ba318db062d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:55.251980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d47d6-1325-481c-a935-2ba318db062d service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:55.262867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 772/1527] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:19:59.657094 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c30c15-66d5-497e-a3de-ee1ba21fbbe9 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:59.657662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c30c15-66d5-497e-a3de-ee1ba21fbbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:59.657758 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c30c15-66d5-497e-a3de-ee1ba21fbbe9 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:59.659147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:19:59.665445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112498d-adce-4943-b279-362f68c4d087 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:59.665973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112498d-adce-4943-b279-362f68c4d087 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:59.666120 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5112498d-adce-4943-b279-362f68c4d087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:59.685100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-5112498d-adce-4943-b279-362f68c4d087 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:19:59.686317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5112498d-adce-4943-b279-362f68c4d087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:59.694771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:19:59.733812 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c30c15-66d5-497e-a3de-ee1ba21fbbe9 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:19:59.735774 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-09c30c15-66d5-497e-a3de-ee1ba21fbbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:19:59.797005 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c30c15-66d5-497e-a3de-ee1ba21fbbe9 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:19:59.800692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd605b-e7fe-4034-a03c-692a28603b75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:19:59.801380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd605b-e7fe-4034-a03c-692a28603b75 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:19:59.801667 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd605b-e7fe-4034-a03c-692a28603b75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:19:59.801945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd605b-e7fe-4034-a03c-692a28603b75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:19:59.812769 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd605b-e7fe-4034-a03c-692a28603b75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:19:59.814299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd605b-e7fe-4034-a03c-692a28603b75 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:19:59.820663 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:19:59.826273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 757/1530] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:19:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:20:02.100333 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4667-ffc6-4335-9f97-c14017a2c03a None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:02.101171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4667-ffc6-4335-9f97-c14017a2c03a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:02.101171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd4667-ffc6-4335-9f97-c14017a2c03a None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:02.102938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:20:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:02.108687 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7a3b4-aab4-4222-8346-9326c322d0ea None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:02.109370 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7a3b4-aab4-4222-8346-9326c322d0ea None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:02.109506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7a3b4-aab4-4222-8346-9326c322d0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:02.129439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-56f7a3b4-aab4-4222-8346-9326c322d0ea None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:02.130895 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-56f7a3b4-aab4-4222-8346-9326c322d0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:02.139711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:20:02.177480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4667-ffc6-4335-9f97-c14017a2c03a None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:02.190604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd4667-ffc6-4335-9f97-c14017a2c03a None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:02.195216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfb70b-c9fc-4ff7-9dc1-4cbc94471f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:02.196066 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfb70b-c9fc-4ff7-9dc1-4cbc94471f88 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:02.196209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfb70b-c9fc-4ff7-9dc1-4cbc94471f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:02.196434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfb70b-c9fc-4ff7-9dc1-4cbc94471f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:02.206923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfb70b-c9fc-4ff7-9dc1-4cbc94471f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:02.208451 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfb70b-c9fc-4ff7-9dc1-4cbc94471f88 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:02.219776 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 775/1533] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:20:06.048694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87bb18-dfa1-4b09-9a72-822b11a80b17 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:06.049296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87bb18-dfa1-4b09-9a72-822b11a80b17 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:06.049424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b87bb18-dfa1-4b09-9a72-822b11a80b17 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:06.050968 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:06.057841 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d7c3c-0348-4b6a-b6fa-e9f32f79acf2 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:06.058420 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d7c3c-0348-4b6a-b6fa-e9f32f79acf2 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:06.058420 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d7c3c-0348-4b6a-b6fa-e9f32f79acf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:06.078468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-063d7c3c-0348-4b6a-b6fa-e9f32f79acf2 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:06.079706 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-063d7c3c-0348-4b6a-b6fa-e9f32f79acf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:06.088402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:20:06.364725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87bb18-dfa1-4b09-9a72-822b11a80b17 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:06.366715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3b87bb18-dfa1-4b09-9a72-822b11a80b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:06.431531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b87bb18-dfa1-4b09-9a72-822b11a80b17 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:06.435290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1645c-238a-47cd-a38a-51323aaaf30e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:06.435955 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1645c-238a-47cd-a38a-51323aaaf30e service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:06.436248 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1645c-238a-47cd-a38a-51323aaaf30e service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:06.436528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1645c-238a-47cd-a38a-51323aaaf30e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:06.447256 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1645c-238a-47cd-a38a-51323aaaf30e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:06.448711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1645c-238a-47cd-a38a-51323aaaf30e service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:06.458829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 760/1536] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:20:08.701571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575aad4b-bf37-4343-88fe-8d450140e35c None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:08.702433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575aad4b-bf37-4343-88fe-8d450140e35c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:08.702605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575aad4b-bf37-4343-88fe-8d450140e35c None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:08.704405 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 15 10:20:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:20:08.711790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370186b-00e6-41d5-8b86-ef2758b17173 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:08.712382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370186b-00e6-41d5-8b86-ef2758b17173 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:08.712382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370186b-00e6-41d5-8b86-ef2758b17173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:08.734302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-f370186b-00e6-41d5-8b86-ef2758b17173 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:20:08.735584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f370186b-00e6-41d5-8b86-ef2758b17173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:08.744117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 15 10:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:20:09.019762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575aad4b-bf37-4343-88fe-8d450140e35c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:20:09.021597 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-575aad4b-bf37-4343-88fe-8d450140e35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:20:09.030919 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=65 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:09.037541 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575aad4b-bf37-4343-88fe-8d450140e35c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:20:09.041395 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c3c6e-2e72-42a6-bf00-fc8a0542e514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:20:09.042065 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c3c6e-2e72-42a6-bf00-fc8a0542e514 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:20:09.042347 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c3c6e-2e72-42a6-bf00-fc8a0542e514 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:20:09.042625 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c3c6e-2e72-42a6-bf00-fc8a0542e514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:20:09.052915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c3c6e-2e72-42a6-bf00-fc8a0542e514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:20:09.054698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c3c6e-2e72-42a6-bf00-fc8a0542e514 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:20:09.070528 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 778/1539] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:20:18.726749 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=32 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:20:19.098514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=5 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:40.940039 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026f317-defc-424e-9b90-9e201e5a94d9 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:40.942229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026f317-defc-424e-9b90-9e201e5a94d9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:40.942229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026f317-defc-424e-9b90-9e201e5a94d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:40.978214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-4026f317-defc-424e-9b90-9e201e5a94d9 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:40.979484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4026f317-defc-424e-9b90-9e201e5a94d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:40.991793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 762/1540] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:21:41.002694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374a875-3727-4229-87e3-59f4a4d04609 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.003807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374a875-3727-4229-87e3-59f4a4d04609 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.003807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374a875-3727-4229-87e3-59f4a4d04609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.031482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-3374a875-3727-4229-87e3-59f4a4d04609 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:41.032806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-3374a875-3727-4229-87e3-59f4a4d04609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:41.049309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 779/1541] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:21:41.054659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026f317-defc-424e-9b90-9e201e5a94d9 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:41.071171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026f317-defc-424e-9b90-9e201e5a94d9 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:41.081792 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea9a57-11be-476f-bfdd-04d122f801c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:41.084605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea9a57-11be-476f-bfdd-04d122f801c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.084605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea9a57-11be-476f-bfdd-04d122f801c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.084605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea9a57-11be-476f-bfdd-04d122f801c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.084605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea9a57-11be-476f-bfdd-04d122f801c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:41.084605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea9a57-11be-476f-bfdd-04d122f801c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:41.085726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ea9a57-11be-476f-bfdd-04d122f801c4 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:41.086513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6cc6f-389f-47a1-b357-e10c79756731 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.086809 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6cc6f-389f-47a1-b357-e10c79756731 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.087089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6cc6f-389f-47a1-b357-e10c79756731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.102653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 763/1542] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:41.104010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc429188-0859-4cf4-a026-3318b7fd6b53 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.104304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc429188-0859-4cf4-a026-3318b7fd6b53 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.104588 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc429188-0859-4cf4-a026-3318b7fd6b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.111246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-d2a6cc6f-389f-47a1-b357-e10c79756731 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:41.112500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d2a6cc6f-389f-47a1-b357-e10c79756731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:41.125640 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 780/1543] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:21:41.129296 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6cc6f-389f-47a1-b357-e10c79756731 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:41.143627 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6cc6f-389f-47a1-b357-e10c79756731 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:41.146057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-bc429188-0859-4cf4-a026-3318b7fd6b53 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:41.147296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-bc429188-0859-4cf4-a026-3318b7fd6b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:41.151610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5d22a-3ec9-4ba3-802e-d542e3cfdaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:41.152283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5d22a-3ec9-4ba3-802e-d542e3cfdaf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.152619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5d22a-3ec9-4ba3-802e-d542e3cfdaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.153358 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5d22a-3ec9-4ba3-802e-d542e3cfdaf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.154056 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5d22a-3ec9-4ba3-802e-d542e3cfdaf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:41.155985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5d22a-3ec9-4ba3-802e-d542e3cfdaf6 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:41.178456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 764/1544] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:41.179790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077278c-ad8a-4548-8bb8-01db470b7a8f None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.180082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077278c-ad8a-4548-8bb8-01db470b7a8f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.180359 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077278c-ad8a-4548-8bb8-01db470b7a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.211666 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-2077278c-ad8a-4548-8bb8-01db470b7a8f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:41.212896 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-2077278c-ad8a-4548-8bb8-01db470b7a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:41.225943 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 765/1545] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:41.227072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d321c-8c92-4a05-8a18-e18255c7cb3c None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.233745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d321c-8c92-4a05-8a18-e18255c7cb3c None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.233745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d321c-8c92-4a05-8a18-e18255c7cb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.250161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-070d321c-8c92-4a05-8a18-e18255c7cb3c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:41.251413 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-070d321c-8c92-4a05-8a18-e18255c7cb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:41.262808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 781/1546] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:21:41.264792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28a63d-8e4f-4b18-b08b-696dd0304ff1 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.267966 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 766/1547] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:41.268698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28a63d-8e4f-4b18-b08b-696dd0304ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.269898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28a63d-8e4f-4b18-b08b-696dd0304ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.270911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a452fb-7c3b-4ce1-a7bd-1d3293454e66 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.271413 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a452fb-7c3b-4ce1-a7bd-1d3293454e66 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.272119 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a452fb-7c3b-4ce1-a7bd-1d3293454e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.292763 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-75a452fb-7c3b-4ce1-a7bd-1d3293454e66 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:41.294200 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-8d28a63d-8e4f-4b18-b08b-696dd0304ff1 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:41.294573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-75a452fb-7c3b-4ce1-a7bd-1d3293454e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:41.295602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-8d28a63d-8e4f-4b18-b08b-696dd0304ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:41.376698 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 767/1548] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:41.378474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a452fb-7c3b-4ce1-a7bd-1d3293454e66 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:41.386677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 782/1549] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:41.391674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248fbe9-db05-4b41-905e-86784dffd49b None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.391674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248fbe9-db05-4b41-905e-86784dffd49b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.391674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248fbe9-db05-4b41-905e-86784dffd49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.391948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a452fb-7c3b-4ce1-a7bd-1d3293454e66 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:41.397102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a06b7-c274-4d78-9822-a764d6c04c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:41.398798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a06b7-c274-4d78-9822-a764d6c04c6c admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.399159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a06b7-c274-4d78-9822-a764d6c04c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.399514 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a06b7-c274-4d78-9822-a764d6c04c6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.400498 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a06b7-c274-4d78-9822-a764d6c04c6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:41.400878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a06b7-c274-4d78-9822-a764d6c04c6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:41.403156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a06b7-c274-4d78-9822-a764d6c04c6c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:41.411989 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-b248fbe9-db05-4b41-905e-86784dffd49b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:41.413381 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b248fbe9-db05-4b41-905e-86784dffd49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:41.418019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 768/1550] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:41.421885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a06b7-c274-4d78-9822-a764d6c04c6c admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:41.428138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 783/1551] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:41.429885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248fbe9-db05-4b41-905e-86784dffd49b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:41.433316 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a06b7-c274-4d78-9822-a764d6c04c6c admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:41.443929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248fbe9-db05-4b41-905e-86784dffd49b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:41.446587 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303f6a8-e126-40a0-86b7-81765ac7685c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:41.447273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303f6a8-e126-40a0-86b7-81765ac7685c admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.447560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303f6a8-e126-40a0-86b7-81765ac7685c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.447834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303f6a8-e126-40a0-86b7-81765ac7685c admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.448535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303f6a8-e126-40a0-86b7-81765ac7685c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:41.448928 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d877f-28f8-4906-8d85-979814fbd904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:41.449624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d877f-28f8-4906-8d85-979814fbd904 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.449907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d877f-28f8-4906-8d85-979814fbd904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.450230 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d877f-28f8-4906-8d85-979814fbd904 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.450489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8303f6a8-e126-40a0-86b7-81765ac7685c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:41.450967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d877f-28f8-4906-8d85-979814fbd904 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:41.451275 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d877f-28f8-4906-8d85-979814fbd904 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:41.452293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: WARNING py.warnings [None req-2a9d877f-28f8-4906-8d85-979814fbd904 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:21:41.452293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: warnings.warn(msg) Jun 15 10:21:41.452293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: Jun 15 10:21:41.453203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d877f-28f8-4906-8d85-979814fbd904 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:41.471984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 784/1552] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:41.471984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c10dab-04aa-4c60-8d5a-8aed3ff51ad5 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.471984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c10dab-04aa-4c60-8d5a-8aed3ff51ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.471984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c10dab-04aa-4c60-8d5a-8aed3ff51ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.488176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-e9c10dab-04aa-4c60-8d5a-8aed3ff51ad5 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:41.489583 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e9c10dab-04aa-4c60-8d5a-8aed3ff51ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:41.497725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 785/1553] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:41.500706 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f893c4-2fb0-434f-a410-a417fe317fee None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.500706 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f893c4-2fb0-434f-a410-a417fe317fee None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.500706 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f893c4-2fb0-434f-a410-a417fe317fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.523708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-e4f893c4-2fb0-434f-a410-a417fe317fee None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:41.523708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e4f893c4-2fb0-434f-a410-a417fe317fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:41.530182 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 786/1554] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:41.531661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c9d74-7588-4ea6-8438-54c4100c0265 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.531950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c9d74-7588-4ea6-8438-54c4100c0265 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.532231 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c9d74-7588-4ea6-8438-54c4100c0265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.552256 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-e11c9d74-7588-4ea6-8438-54c4100c0265 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:41.553479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e11c9d74-7588-4ea6-8438-54c4100c0265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:41.561859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 787/1555] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:41.567278 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c9d74-7588-4ea6-8438-54c4100c0265 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:41.573857 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 769/1556] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:41.575102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b5ac4-cd7d-4142-b291-58ef5729f4fc None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.575404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b5ac4-cd7d-4142-b291-58ef5729f4fc None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.575689 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b5ac4-cd7d-4142-b291-58ef5729f4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.579982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c9d74-7588-4ea6-8438-54c4100c0265 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:41.585603 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de8aaf-0ae2-4078-b2e0-4d77144e8fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:41.586280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de8aaf-0ae2-4078-b2e0-4d77144e8fec admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.586558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de8aaf-0ae2-4078-b2e0-4d77144e8fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.586829 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de8aaf-0ae2-4078-b2e0-4d77144e8fec admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.587520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de8aaf-0ae2-4078-b2e0-4d77144e8fec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:41.589526 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de8aaf-0ae2-4078-b2e0-4d77144e8fec admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:41.596386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b63b5ac4-cd7d-4142-b291-58ef5729f4fc None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:41.598109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b63b5ac4-cd7d-4142-b291-58ef5729f4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:41.606854 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 770/1557] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:41.608197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d893b-a7d4-436f-b98b-702fb09eb490 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.611637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d893b-a7d4-436f-b98b-702fb09eb490 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.611637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d893b-a7d4-436f-b98b-702fb09eb490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.626782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-ee2d893b-a7d4-436f-b98b-702fb09eb490 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:41.627973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ee2d893b-a7d4-436f-b98b-702fb09eb490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:41.636063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 771/1558] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:41.636825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d893b-a7d4-436f-b98b-702fb09eb490 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:41.647287 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d893b-a7d4-436f-b98b-702fb09eb490 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:41.655347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e3bef-f0bf-4dbd-9b00-fd16646a23cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:41.655347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e3bef-f0bf-4dbd-9b00-fd16646a23cd admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.655347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e3bef-f0bf-4dbd-9b00-fd16646a23cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.655347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e3bef-f0bf-4dbd-9b00-fd16646a23cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.655909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e3bef-f0bf-4dbd-9b00-fd16646a23cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:41.656927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: WARNING py.warnings [None req-1a1e3bef-f0bf-4dbd-9b00-fd16646a23cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:21:41.656927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: warnings.warn(msg) Jun 15 10:21:41.656927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: Jun 15 10:21:41.658154 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1e3bef-f0bf-4dbd-9b00-fd16646a23cd admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:41.697676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 788/1559] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:41.699120 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe21bc-57cb-4a0e-aece-31e17b72fe9d None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.699416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe21bc-57cb-4a0e-aece-31e17b72fe9d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.699707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe21bc-57cb-4a0e-aece-31e17b72fe9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.718314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-42fe21bc-57cb-4a0e-aece-31e17b72fe9d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:41.719515 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-42fe21bc-57cb-4a0e-aece-31e17b72fe9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:41.772591 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 772/1560] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:41.773612 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e3bef-f0bf-4dbd-9b00-fd16646a23cd admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:41.783246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 789/1561] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:41.784143 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe21bc-57cb-4a0e-aece-31e17b72fe9d None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:41.791839 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:41.795985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe21bc-57cb-4a0e-aece-31e17b72fe9d None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:41.837672 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:41.843098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1e3bef-f0bf-4dbd-9b00-fd16646a23cd admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:41.849727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453458b-c54e-4c62-8144-35c07ee6f7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:41.850431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453458b-c54e-4c62-8144-35c07ee6f7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.850718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453458b-c54e-4c62-8144-35c07ee6f7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.851005 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453458b-c54e-4c62-8144-35c07ee6f7b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.851748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453458b-c54e-4c62-8144-35c07ee6f7b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:41.852065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453458b-c54e-4c62-8144-35c07ee6f7b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:41.853755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453458b-c54e-4c62-8144-35c07ee6f7b5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:41.858960 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606cc33-007b-428c-b587-49df7cc22610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:41.862491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606cc33-007b-428c-b587-49df7cc22610 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.862491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606cc33-007b-428c-b587-49df7cc22610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.862491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0606cc33-007b-428c-b587-49df7cc22610 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.862491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606cc33-007b-428c-b587-49df7cc22610 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:41.862491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606cc33-007b-428c-b587-49df7cc22610 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:41.863001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0606cc33-007b-428c-b587-49df7cc22610 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:41.873609 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 773/1562] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:41.873609 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453458b-c54e-4c62-8144-35c07ee6f7b5 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:41.876944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 790/1563] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:41.877692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606cc33-007b-428c-b587-49df7cc22610 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:41.888928 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0606cc33-007b-428c-b587-49df7cc22610 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:41.890136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453458b-c54e-4c62-8144-35c07ee6f7b5 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:41.893914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c29ada-53a6-4da7-b709-028f84a20316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:41.895152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c29ada-53a6-4da7-b709-028f84a20316 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.895442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c29ada-53a6-4da7-b709-028f84a20316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.895724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c29ada-53a6-4da7-b709-028f84a20316 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.896466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c29ada-53a6-4da7-b709-028f84a20316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:41.896580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b2cbb-8432-4fc1-aa79-fbe6db6937b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:41.896777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c29ada-53a6-4da7-b709-028f84a20316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:41.897447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b2cbb-8432-4fc1-aa79-fbe6db6937b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.897739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b2cbb-8432-4fc1-aa79-fbe6db6937b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.898017 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b2cbb-8432-4fc1-aa79-fbe6db6937b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.898570 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c29ada-53a6-4da7-b709-028f84a20316 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:41.898755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b2cbb-8432-4fc1-aa79-fbe6db6937b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:41.900356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b2cbb-8432-4fc1-aa79-fbe6db6937b2 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:41.914146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 791/1565] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:41.914146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992371e5-500e-4839-a9ce-bc4180371761 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.914574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 774/1565] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:41.914742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992371e5-500e-4839-a9ce-bc4180371761 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.914742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992371e5-500e-4839-a9ce-bc4180371761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.915039 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b2cbb-8432-4fc1-aa79-fbe6db6937b2 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:41.926848 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b2cbb-8432-4fc1-aa79-fbe6db6937b2 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:41.932076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274fc52-3d5d-4445-80c0-4f6d9b5813ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:41.932446 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-992371e5-500e-4839-a9ce-bc4180371761 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:41.932842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274fc52-3d5d-4445-80c0-4f6d9b5813ab admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.933254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274fc52-3d5d-4445-80c0-4f6d9b5813ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.933538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274fc52-3d5d-4445-80c0-4f6d9b5813ab admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.933697 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-992371e5-500e-4839-a9ce-bc4180371761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:41.934236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274fc52-3d5d-4445-80c0-4f6d9b5813ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:41.938590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274fc52-3d5d-4445-80c0-4f6d9b5813ab admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:41.942662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 792/1566] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:41.943452 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992371e5-500e-4839-a9ce-bc4180371761 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:41.953451 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992371e5-500e-4839-a9ce-bc4180371761 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:41.959083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c26d5d-b82e-461a-b28a-72bb2b751fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:41.965324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c26d5d-b82e-461a-b28a-72bb2b751fcd admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:41.965324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c26d5d-b82e-461a-b28a-72bb2b751fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:41.965324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c26d5d-b82e-461a-b28a-72bb2b751fcd admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:41.965324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c26d5d-b82e-461a-b28a-72bb2b751fcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:41.965324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: WARNING py.warnings [None req-97c26d5d-b82e-461a-b28a-72bb2b751fcd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:21:41.965324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: warnings.warn(msg) Jun 15 10:21:41.965324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: Jun 15 10:21:41.966192 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c26d5d-b82e-461a-b28a-72bb2b751fcd admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.048311 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 775/1567] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/users => generated 285 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:42.049138 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274fc52-3d5d-4445-80c0-4f6d9b5813ab admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.059432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274fc52-3d5d-4445-80c0-4f6d9b5813ab admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.063021 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 793/1568] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:21:41 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:42.063975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c26d5d-b82e-461a-b28a-72bb2b751fcd admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.064488 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f31a63-95a6-4f20-b4fa-efdd0da9566f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.065176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f31a63-95a6-4f20-b4fa-efdd0da9566f admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.065509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f31a63-95a6-4f20-b4fa-efdd0da9566f admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.066633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f31a63-95a6-4f20-b4fa-efdd0da9566f admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.067433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f31a63-95a6-4f20-b4fa-efdd0da9566f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:42.067758 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f31a63-95a6-4f20-b4fa-efdd0da9566f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.069418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f31a63-95a6-4f20-b4fa-efdd0da9566f admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.074904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c26d5d-b82e-461a-b28a-72bb2b751fcd admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.079654 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb67be0-1a7b-4b44-8cdf-b2feda89e0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.093500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb67be0-1a7b-4b44-8cdf-b2feda89e0c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.093500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb67be0-1a7b-4b44-8cdf-b2feda89e0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.093500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb67be0-1a7b-4b44-8cdf-b2feda89e0c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.093500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb67be0-1a7b-4b44-8cdf-b2feda89e0c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.094296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 776/1569] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:42.094296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f31a63-95a6-4f20-b4fa-efdd0da9566f admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.094609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb67be0-1a7b-4b44-8cdf-b2feda89e0c4 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.094738 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f31a63-95a6-4f20-b4fa-efdd0da9566f admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.099143 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c09dd-291e-4e72-ae70-3bed53d0dea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.099812 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c09dd-291e-4e72-ae70-3bed53d0dea0 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.100090 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c09dd-291e-4e72-ae70-3bed53d0dea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.100365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c09dd-291e-4e72-ae70-3bed53d0dea0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.101128 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c09dd-291e-4e72-ae70-3bed53d0dea0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.103008 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11c09dd-291e-4e72-ae70-3bed53d0dea0 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.169536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 794/1570] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:21:42 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:42.170392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb67be0-1a7b-4b44-8cdf-b2feda89e0c4 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.200543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 777/1571] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:21:42 2022] POST /identity/v3/projects => generated 389 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:42.201339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c09dd-291e-4e72-ae70-3bed53d0dea0 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.236246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb67be0-1a7b-4b44-8cdf-b2feda89e0c4 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.269360 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c09dd-291e-4e72-ae70-3bed53d0dea0 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.274221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cb04a-348b-4d59-b505-a73943a80687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.274901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cb04a-348b-4d59-b505-a73943a80687 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.275185 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cb04a-348b-4d59-b505-a73943a80687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.275461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cb04a-348b-4d59-b505-a73943a80687 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.276153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502cb04a-348b-4d59-b505-a73943a80687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.278124 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502cb04a-348b-4d59-b505-a73943a80687 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.285519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:42.297986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d75b60-69e4-4b9e-82d5-78c44e5aa209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.298706 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d75b60-69e4-4b9e-82d5-78c44e5aa209 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.298936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d75b60-69e4-4b9e-82d5-78c44e5aa209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.299213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d75b60-69e4-4b9e-82d5-78c44e5aa209 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.299948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d75b60-69e4-4b9e-82d5-78c44e5aa209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.302181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d75b60-69e4-4b9e-82d5-78c44e5aa209 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.312200 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 795/1572] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:42.313028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d75b60-69e4-4b9e-82d5-78c44e5aa209 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.324140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d75b60-69e4-4b9e-82d5-78c44e5aa209 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.328894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6f3f2-b3ba-4940-86c0-b5fefa4693fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.329578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6f3f2-b3ba-4940-86c0-b5fefa4693fa admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.329859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6f3f2-b3ba-4940-86c0-b5fefa4693fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.330138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6f3f2-b3ba-4940-86c0-b5fefa4693fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.330858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6f3f2-b3ba-4940-86c0-b5fefa4693fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:42.331233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6f3f2-b3ba-4940-86c0-b5fefa4693fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.332986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6f3f2-b3ba-4940-86c0-b5fefa4693fa admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.345404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 796/1573] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:42.350317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6f3f2-b3ba-4940-86c0-b5fefa4693fa admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.356041 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6f3f2-b3ba-4940-86c0-b5fefa4693fa admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.360899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb96c0-7a90-4ec0-bbe3-e87e1f54d948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.361568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb96c0-7a90-4ec0-bbe3-e87e1f54d948 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.361959 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb96c0-7a90-4ec0-bbe3-e87e1f54d948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.362237 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb96c0-7a90-4ec0-bbe3-e87e1f54d948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.363959 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb96c0-7a90-4ec0-bbe3-e87e1f54d948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.365106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb96c0-7a90-4ec0-bbe3-e87e1f54d948 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.376087 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 778/1574] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:21:42 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:42.381586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 797/1575] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:42.381741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cb04a-348b-4d59-b505-a73943a80687 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.381858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb96c0-7a90-4ec0-bbe3-e87e1f54d948 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.388113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb96c0-7a90-4ec0-bbe3-e87e1f54d948 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.447968 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cb04a-348b-4d59-b505-a73943a80687 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.450314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c7ab1-989f-450b-9dad-75f57973855a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.451008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c7ab1-989f-450b-9dad-75f57973855a admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.451299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c7ab1-989f-450b-9dad-75f57973855a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.451640 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c7ab1-989f-450b-9dad-75f57973855a admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.452386 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c7ab1-989f-450b-9dad-75f57973855a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.453026 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956e9b5-dd04-4db8-a74c-f568475fb8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.453728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956e9b5-dd04-4db8-a74c-f568475fb8db admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.454015 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956e9b5-dd04-4db8-a74c-f568475fb8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.454291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956e9b5-dd04-4db8-a74c-f568475fb8db admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.454387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c7ab1-989f-450b-9dad-75f57973855a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.455062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956e9b5-dd04-4db8-a74c-f568475fb8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.457161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956e9b5-dd04-4db8-a74c-f568475fb8db admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.467306 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 779/1576] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:42.468132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956e9b5-dd04-4db8-a74c-f568475fb8db admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.478679 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956e9b5-dd04-4db8-a74c-f568475fb8db admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.483314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34537c-ec96-471e-8493-67d90da8ace6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.483999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34537c-ec96-471e-8493-67d90da8ace6 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.484265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34537c-ec96-471e-8493-67d90da8ace6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.484542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d34537c-ec96-471e-8493-67d90da8ace6 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.485283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d34537c-ec96-471e-8493-67d90da8ace6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.487203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d34537c-ec96-471e-8493-67d90da8ace6 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.494657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:42.538231 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 798/1577] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:21:42 2022] POST /identity/v3/projects => generated 389 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:42.539210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c7ab1-989f-450b-9dad-75f57973855a admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.568466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 780/1578] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:21:42 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:42.568466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34537c-ec96-471e-8493-67d90da8ace6 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.578797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d34537c-ec96-471e-8493-67d90da8ace6 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.613460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c7ab1-989f-450b-9dad-75f57973855a admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.619818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882212f-d2a0-49f0-8a21-826a1fe38ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.620828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882212f-d2a0-49f0-8a21-826a1fe38ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.621249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882212f-d2a0-49f0-8a21-826a1fe38ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.621677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882212f-d2a0-49f0-8a21-826a1fe38ab5 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.622534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882212f-d2a0-49f0-8a21-826a1fe38ab5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.625089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882212f-d2a0-49f0-8a21-826a1fe38ab5 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.647364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30ca94-dcc1-4843-8d13-87964c2a5a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.648267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30ca94-dcc1-4843-8d13-87964c2a5a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.648549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30ca94-dcc1-4843-8d13-87964c2a5a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.648827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30ca94-dcc1-4843-8d13-87964c2a5a01 admin admin] PATH_INFO: `/v3/projects/0b8e3aa03dcf4c5a989a604548e9c4ba/users/3f8028df66d44c478268a84167c9ff19/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.663944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30ca94-dcc1-4843-8d13-87964c2a5a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8e3aa03dcf4c5a989a604548e9c4ba, user_id=3f8028df66d44c478268a84167c9ff19, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.665009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: WARNING py.warnings [None req-1d30ca94-dcc1-4843-8d13-87964c2a5a01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:21:42.665009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: warnings.warn(msg) Jun 15 10:21:42.665009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: Jun 15 10:21:42.666973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d30ca94-dcc1-4843-8d13-87964c2a5a01 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.730893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 799/1579] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:21:42 2022] POST /identity/v3/users => generated 295 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:42.731717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882212f-d2a0-49f0-8a21-826a1fe38ab5 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.742493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882212f-d2a0-49f0-8a21-826a1fe38ab5 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.747115 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09af236-8254-42c4-a626-0ed786b02abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.747760 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09af236-8254-42c4-a626-0ed786b02abb admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.748043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09af236-8254-42c4-a626-0ed786b02abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.748319 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09af236-8254-42c4-a626-0ed786b02abb admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.749059 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09af236-8254-42c4-a626-0ed786b02abb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.750978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09af236-8254-42c4-a626-0ed786b02abb admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.773747 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 781/1580] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:42 2022] PUT /identity/v3/projects/0b8e3aa03dcf4c5a989a604548e9c4ba/users/3f8028df66d44c478268a84167c9ff19/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:42.774649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30ca94-dcc1-4843-8d13-87964c2a5a01 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.835808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 800/1581] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:21:42 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:42.836593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09af236-8254-42c4-a626-0ed786b02abb admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.841892 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30ca94-dcc1-4843-8d13-87964c2a5a01 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.849947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:42.901576 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083334de-44ea-4466-845b-32b923c46813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.901862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09af236-8254-42c4-a626-0ed786b02abb admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.902251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083334de-44ea-4466-845b-32b923c46813 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.902534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083334de-44ea-4466-845b-32b923c46813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.902810 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083334de-44ea-4466-845b-32b923c46813 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.903503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083334de-44ea-4466-845b-32b923c46813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.905449 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083334de-44ea-4466-845b-32b923c46813 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.907515 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc33a56-c78b-4669-8bbb-5a5d97c189d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.908581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc33a56-c78b-4669-8bbb-5a5d97c189d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.908983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc33a56-c78b-4669-8bbb-5a5d97c189d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.909363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc33a56-c78b-4669-8bbb-5a5d97c189d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.910801 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc33a56-c78b-4669-8bbb-5a5d97c189d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.912589 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc33a56-c78b-4669-8bbb-5a5d97c189d9 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.923261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 801/1582] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:42.924099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc33a56-c78b-4669-8bbb-5a5d97c189d9 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.934041 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc33a56-c78b-4669-8bbb-5a5d97c189d9 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.938604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b4d0e-8040-4bab-ba00-c5ec94818b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.939255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b4d0e-8040-4bab-ba00-c5ec94818b25 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.939541 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b4d0e-8040-4bab-ba00-c5ec94818b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.939821 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b4d0e-8040-4bab-ba00-c5ec94818b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.940551 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b4d0e-8040-4bab-ba00-c5ec94818b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.942524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b4d0e-8040-4bab-ba00-c5ec94818b25 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:42.958479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 802/1583] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:42.959299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b4d0e-8040-4bab-ba00-c5ec94818b25 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:42.968784 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b4d0e-8040-4bab-ba00-c5ec94818b25 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:42.973954 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9d188-4969-484e-9d18-a4a204a8f5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:42.974622 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9d188-4969-484e-9d18-a4a204a8f5b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:42.974905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9d188-4969-484e-9d18-a4a204a8f5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:42.975204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9d188-4969-484e-9d18-a4a204a8f5b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:42.975873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9d188-4969-484e-9d18-a4a204a8f5b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:42.977922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9d188-4969-484e-9d18-a4a204a8f5b7 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.007916 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 782/1584] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:21:42 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:43.008676 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083334de-44ea-4466-845b-32b923c46813 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.018223 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083334de-44ea-4466-845b-32b923c46813 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.023752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2321a80-b01e-403f-a540-5ed35de09b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.024361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2321a80-b01e-403f-a540-5ed35de09b80 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.024643 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2321a80-b01e-403f-a540-5ed35de09b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.024925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2321a80-b01e-403f-a540-5ed35de09b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.025690 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2321a80-b01e-403f-a540-5ed35de09b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.027289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2321a80-b01e-403f-a540-5ed35de09b80 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.029688 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:43.038647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 783/1585] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:43.039456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2321a80-b01e-403f-a540-5ed35de09b80 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.048985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2321a80-b01e-403f-a540-5ed35de09b80 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.054059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2247b-37e5-4602-b794-4f1765b9891e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.054725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2247b-37e5-4602-b794-4f1765b9891e admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.055010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2247b-37e5-4602-b794-4f1765b9891e admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.055286 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2247b-37e5-4602-b794-4f1765b9891e admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.056187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2247b-37e5-4602-b794-4f1765b9891e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.057662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2247b-37e5-4602-b794-4f1765b9891e admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.062043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 803/1586] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:21:42 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:43.065238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3897746-b77c-4931-8992-0bd37c05a051 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.065238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3897746-b77c-4931-8992-0bd37c05a051 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.065238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3897746-b77c-4931-8992-0bd37c05a051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.075844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 784/1587] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:43.076619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2247b-37e5-4602-b794-4f1765b9891e admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.083688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-b3897746-b77c-4931-8992-0bd37c05a051 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:43.084898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b3897746-b77c-4931-8992-0bd37c05a051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:43.087038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2247b-37e5-4602-b794-4f1765b9891e admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.091732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ae93c-e587-46f1-81df-0b69b8d15a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.092572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 804/1588] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:43.092702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ae93c-e587-46f1-81df-0b69b8d15a25 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.092928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ae93c-e587-46f1-81df-0b69b8d15a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.093241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ae93c-e587-46f1-81df-0b69b8d15a25 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.093399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3897746-b77c-4931-8992-0bd37c05a051 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.093944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ae93c-e587-46f1-81df-0b69b8d15a25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.096235 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ae93c-e587-46f1-81df-0b69b8d15a25 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.104320 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3897746-b77c-4931-8992-0bd37c05a051 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.109125 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87eced-eec3-4c96-825d-3d1c6c527964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.110026 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87eced-eec3-4c96-825d-3d1c6c527964 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.110299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87eced-eec3-4c96-825d-3d1c6c527964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.110573 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87eced-eec3-4c96-825d-3d1c6c527964 admin admin] PATH_INFO: `/v3/projects/8e28d54c66fd43b7bd68bd5a95f5df4d/users/f5ca2c3d2a964f959d50eba45287184c/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.126239 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87eced-eec3-4c96-825d-3d1c6c527964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e28d54c66fd43b7bd68bd5a95f5df4d, user_id=f5ca2c3d2a964f959d50eba45287184c, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.129601 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87eced-eec3-4c96-825d-3d1c6c527964 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.196795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 785/1589] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:21:43 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:43.197666 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ae93c-e587-46f1-81df-0b69b8d15a25 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.208933 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ae93c-e587-46f1-81df-0b69b8d15a25 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.211493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 805/1590] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:43 2022] PUT /identity/v3/projects/8e28d54c66fd43b7bd68bd5a95f5df4d/users/f5ca2c3d2a964f959d50eba45287184c/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:43.218532 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87eced-eec3-4c96-825d-3d1c6c527964 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.278221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75757c-3457-4e18-8ddc-f857d1c33f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.279122 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75757c-3457-4e18-8ddc-f857d1c33f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.279405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75757c-3457-4e18-8ddc-f857d1c33f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.279675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75757c-3457-4e18-8ddc-f857d1c33f3e admin admin] PATH_INFO: `/v3/projects/a09fe8836f914998aec8ee2afcca9e16/users/9f1e192f3eab42ee9f9ed5d2789e0d1e/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.289535 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87eced-eec3-4c96-825d-3d1c6c527964 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.296139 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75757c-3457-4e18-8ddc-f857d1c33f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a09fe8836f914998aec8ee2afcca9e16, user_id=9f1e192f3eab42ee9f9ed5d2789e0d1e, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.296536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9cc0f-975a-49ec-9aa9-b1de23ce78dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.297396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9cc0f-975a-49ec-9aa9-b1de23ce78dc admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.297838 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9cc0f-975a-49ec-9aa9-b1de23ce78dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.298268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9cc0f-975a-49ec-9aa9-b1de23ce78dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.298878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75757c-3457-4e18-8ddc-f857d1c33f3e admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.299209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9cc0f-975a-49ec-9aa9-b1de23ce78dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.301100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf9cc0f-975a-49ec-9aa9-b1de23ce78dc admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.311873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 806/1591] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:43.317294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9cc0f-975a-49ec-9aa9-b1de23ce78dc admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.324090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9cc0f-975a-49ec-9aa9-b1de23ce78dc admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.329008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b67aa-6dbd-4257-b162-e3b3c1d56d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.329686 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b67aa-6dbd-4257-b162-e3b3c1d56d8f admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.329973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b67aa-6dbd-4257-b162-e3b3c1d56d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.330254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b67aa-6dbd-4257-b162-e3b3c1d56d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.330987 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b67aa-6dbd-4257-b162-e3b3c1d56d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.333209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b67aa-6dbd-4257-b162-e3b3c1d56d8f admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.347721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 807/1592] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:43.347721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b67aa-6dbd-4257-b162-e3b3c1d56d8f admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.354461 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b67aa-6dbd-4257-b162-e3b3c1d56d8f admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.359801 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6af9d-3a8d-43bf-8017-fc2dea49c389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.360476 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6af9d-3a8d-43bf-8017-fc2dea49c389 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.360758 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6af9d-3a8d-43bf-8017-fc2dea49c389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.361093 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6af9d-3a8d-43bf-8017-fc2dea49c389 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.361828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6af9d-3a8d-43bf-8017-fc2dea49c389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.363518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6af9d-3a8d-43bf-8017-fc2dea49c389 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.374311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 808/1593] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:43.376165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6af9d-3a8d-43bf-8017-fc2dea49c389 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.385586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6af9d-3a8d-43bf-8017-fc2dea49c389 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.390489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d513c9-3e74-409b-a8d7-ab597589e9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.391140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d513c9-3e74-409b-a8d7-ab597589e9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.391421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d513c9-3e74-409b-a8d7-ab597589e9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.391702 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d513c9-3e74-409b-a8d7-ab597589e9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.392430 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d513c9-3e74-409b-a8d7-ab597589e9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.394699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d513c9-3e74-409b-a8d7-ab597589e9a2 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.403195 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 786/1594] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:43 2022] PUT /identity/v3/projects/a09fe8836f914998aec8ee2afcca9e16/users/9f1e192f3eab42ee9f9ed5d2789e0d1e/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:43.407418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75757c-3457-4e18-8ddc-f857d1c33f3e admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.407571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 809/1595] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:43.407571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d513c9-3e74-409b-a8d7-ab597589e9a2 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.409458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1c75757c-3457-4e18-8ddc-f857d1c33f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:43.416823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d513c9-3e74-409b-a8d7-ab597589e9a2 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.425673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:43.476719 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4481a6-e78f-4cda-9d45-180129eaa96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.477090 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75757c-3457-4e18-8ddc-f857d1c33f3e admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.477428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4481a6-e78f-4cda-9d45-180129eaa96d admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.477873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4481a6-e78f-4cda-9d45-180129eaa96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.478194 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4481a6-e78f-4cda-9d45-180129eaa96d admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.478895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4481a6-e78f-4cda-9d45-180129eaa96d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.480556 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4481a6-e78f-4cda-9d45-180129eaa96d admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.480758 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438af156-9a04-4667-bae0-9452d4dd7103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.481480 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438af156-9a04-4667-bae0-9452d4dd7103 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.481737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438af156-9a04-4667-bae0-9452d4dd7103 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.482012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438af156-9a04-4667-bae0-9452d4dd7103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.491542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 810/1596] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:43.492651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438af156-9a04-4667-bae0-9452d4dd7103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.492824 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694e092-a397-4283-b3ef-93cb4f816e73 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.493149 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694e092-a397-4283-b3ef-93cb4f816e73 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.493434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694e092-a397-4283-b3ef-93cb4f816e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.494509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438af156-9a04-4667-bae0-9452d4dd7103 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.505868 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 787/1597] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:43.506714 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438af156-9a04-4667-bae0-9452d4dd7103 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.517040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438af156-9a04-4667-bae0-9452d4dd7103 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.522772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-0694e092-a397-4283-b3ef-93cb4f816e73 None None] MFA Rules not processed for user `f5ca2c3d2a964f959d50eba45287184c`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:43.524084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0694e092-a397-4283-b3ef-93cb4f816e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:43.527946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0bf36-9f1b-4386-a818-5694cf800bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.528766 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0bf36-9f1b-4386-a818-5694cf800bca admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.529164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0bf36-9f1b-4386-a818-5694cf800bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.529522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0bf36-9f1b-4386-a818-5694cf800bca admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.530440 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0bf36-9f1b-4386-a818-5694cf800bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.532180 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0bf36-9f1b-4386-a818-5694cf800bca admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.545515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 788/1598] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:43.546428 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0bf36-9f1b-4386-a818-5694cf800bca admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.561773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0bf36-9f1b-4386-a818-5694cf800bca admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.568309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f976de-4763-4e5b-b453-d1306796056f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.569238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f976de-4763-4e5b-b453-d1306796056f admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.569522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f976de-4763-4e5b-b453-d1306796056f admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.571687 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f976de-4763-4e5b-b453-d1306796056f admin admin] PATH_INFO: `/v3/projects/b308be72b7e440058d72246650648cf1/users/7e2d196819c54e049858fc19139a0bda/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.587926 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f976de-4763-4e5b-b453-d1306796056f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b308be72b7e440058d72246650648cf1, user_id=7e2d196819c54e049858fc19139a0bda, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.590776 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f976de-4763-4e5b-b453-d1306796056f admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.597331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 811/1599] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:43.601792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694e092-a397-4283-b3ef-93cb4f816e73 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.613242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694e092-a397-4283-b3ef-93cb4f816e73 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.618259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f7ff47-7c9d-4b44-beec-14308ca1d897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.618831 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7ff47-7c9d-4b44-beec-14308ca1d897 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.619117 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7ff47-7c9d-4b44-beec-14308ca1d897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.619400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f7ff47-7c9d-4b44-beec-14308ca1d897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.620141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7ff47-7c9d-4b44-beec-14308ca1d897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.621911 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f7ff47-7c9d-4b44-beec-14308ca1d897 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.637584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 812/1600] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:43.639675 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf90c86-9e7e-4de5-bb82-3263a815459b None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.639675 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf90c86-9e7e-4de5-bb82-3263a815459b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.639675 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf90c86-9e7e-4de5-bb82-3263a815459b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.658803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-cdf90c86-9e7e-4de5-bb82-3263a815459b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:43.660451 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-cdf90c86-9e7e-4de5-bb82-3263a815459b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:43.667834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 789/1601] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:43 2022] PUT /identity/v3/projects/b308be72b7e440058d72246650648cf1/users/7e2d196819c54e049858fc19139a0bda/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:43.668952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f976de-4763-4e5b-b453-d1306796056f admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.710989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:43.750396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 813/1602] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:43.751731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf90c86-9e7e-4de5-bb82-3263a815459b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.763208 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf90c86-9e7e-4de5-bb82-3263a815459b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.772332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e73a5a-51f0-448a-a55b-79b8fbbb1996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.772872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e73a5a-51f0-448a-a55b-79b8fbbb1996 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.773193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e73a5a-51f0-448a-a55b-79b8fbbb1996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.773471 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e73a5a-51f0-448a-a55b-79b8fbbb1996 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.774266 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e73a5a-51f0-448a-a55b-79b8fbbb1996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.774773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f976de-4763-4e5b-b453-d1306796056f admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.776565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e73a5a-51f0-448a-a55b-79b8fbbb1996 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.781561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab88d0-d85c-411c-aa8f-0d8e83e21ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.783213 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab88d0-d85c-411c-aa8f-0d8e83e21ab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.783762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab88d0-d85c-411c-aa8f-0d8e83e21ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.784135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab88d0-d85c-411c-aa8f-0d8e83e21ab6 admin admin] PATH_INFO: `/v3/projects/bbcfca56769742b9b3645f8f7c9bc6e3/users/bbb69faf3ec54489a92fe87935f2f777/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.788324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 814/1603] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:43.788324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e73a5a-51f0-448a-a55b-79b8fbbb1996 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.799065 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e73a5a-51f0-448a-a55b-79b8fbbb1996 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:43.804873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a001ea6-37bf-43cc-9cb9-2146b612394b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:43.806423 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab88d0-d85c-411c-aa8f-0d8e83e21ab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbcfca56769742b9b3645f8f7c9bc6e3, user_id=bbb69faf3ec54489a92fe87935f2f777, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.806845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a001ea6-37bf-43cc-9cb9-2146b612394b admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:43.807182 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a001ea6-37bf-43cc-9cb9-2146b612394b admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:43.807469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a001ea6-37bf-43cc-9cb9-2146b612394b admin admin] PATH_INFO: `/v3/projects/cb38b3cc097e4d9ba642740438c88dfc/users/4612504f393546dfbea12ffb3eed5efe/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:43.809417 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab88d0-d85c-411c-aa8f-0d8e83e21ab6 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.821106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:43.824483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a001ea6-37bf-43cc-9cb9-2146b612394b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb38b3cc097e4d9ba642740438c88dfc, user_id=4612504f393546dfbea12ffb3eed5efe, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:43.827363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a001ea6-37bf-43cc-9cb9-2146b612394b admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:43.936082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 790/1604] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:43 2022] PUT /identity/v3/projects/bbcfca56769742b9b3645f8f7c9bc6e3/users/bbb69faf3ec54489a92fe87935f2f777/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:43.937155 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab88d0-d85c-411c-aa8f-0d8e83e21ab6 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:43.989204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 815/1605] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:43 2022] PUT /identity/v3/projects/cb38b3cc097e4d9ba642740438c88dfc/users/4612504f393546dfbea12ffb3eed5efe/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:43.990205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a001ea6-37bf-43cc-9cb9-2146b612394b admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:44.017605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab88d0-d85c-411c-aa8f-0d8e83e21ab6 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.072779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a001ea6-37bf-43cc-9cb9-2146b612394b admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.081325 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c62e38-bfca-498c-ae08-2a220e776beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.082072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c62e38-bfca-498c-ae08-2a220e776beb admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.082372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c62e38-bfca-498c-ae08-2a220e776beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.082661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c62e38-bfca-498c-ae08-2a220e776beb admin admin] PATH_INFO: `/v3/projects/ab32692641974828b6fa534495acca01/users/5090fa81be4d41bbb62da2c4f273d537/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.086340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99833dba-29b8-4d9e-863b-3fc307a116ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.087034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99833dba-29b8-4d9e-863b-3fc307a116ec admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.087346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99833dba-29b8-4d9e-863b-3fc307a116ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.087630 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99833dba-29b8-4d9e-863b-3fc307a116ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.088370 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99833dba-29b8-4d9e-863b-3fc307a116ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:44.090366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99833dba-29b8-4d9e-863b-3fc307a116ec admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:44.102506 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c62e38-bfca-498c-ae08-2a220e776beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab32692641974828b6fa534495acca01, user_id=5090fa81be4d41bbb62da2c4f273d537, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:44.105277 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c62e38-bfca-498c-ae08-2a220e776beb admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:44.205358 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 816/1606] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:43 2022] PUT /identity/v3/projects/ab32692641974828b6fa534495acca01/users/5090fa81be4d41bbb62da2c4f273d537/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:44.207003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628646b-ade6-4d57-b6b6-0ce429d578aa None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.207291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628646b-ade6-4d57-b6b6-0ce429d578aa None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.210829 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628646b-ade6-4d57-b6b6-0ce429d578aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.211000 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 791/1607] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:21:43 2022] POST /identity/v3/projects => generated 369 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:44.211000 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99833dba-29b8-4d9e-863b-3fc307a116ec admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:44.239814 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-b628646b-ade6-4d57-b6b6-0ce429d578aa None None] MFA Rules not processed for user `7e2d196819c54e049858fc19139a0bda`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:44.241254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b628646b-ade6-4d57-b6b6-0ce429d578aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:44.253214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:44.271655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99833dba-29b8-4d9e-863b-3fc307a116ec admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.275400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4be724-0569-4edb-9c16-3cd494d1b646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.276050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4be724-0569-4edb-9c16-3cd494d1b646 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.276347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4be724-0569-4edb-9c16-3cd494d1b646 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.276607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4be724-0569-4edb-9c16-3cd494d1b646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.319086 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 817/1608] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:44.319945 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628646b-ade6-4d57-b6b6-0ce429d578aa None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:44.350446 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4be724-0569-4edb-9c16-3cd494d1b646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:44.351911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4be724-0569-4edb-9c16-3cd494d1b646 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:44.365129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 792/1609] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:44.366525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b691f-56b4-4019-9aec-1de41f77df46 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.366820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b691f-56b4-4019-9aec-1de41f77df46 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.367105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b691f-56b4-4019-9aec-1de41f77df46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.385514 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-894b691f-56b4-4019-9aec-1de41f77df46 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:44.387459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-894b691f-56b4-4019-9aec-1de41f77df46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:44.392838 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b628646b-ade6-4d57-b6b6-0ce429d578aa None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.397020 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 793/1610] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:44.398309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade1ff5-5ec7-4f07-a052-89671a096b2e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.398445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe8f89-b6ca-4f6d-b80e-1953766b52e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.398619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade1ff5-5ec7-4f07-a052-89671a096b2e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.399012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade1ff5-5ec7-4f07-a052-89671a096b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.399216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe8f89-b6ca-4f6d-b80e-1953766b52e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.401681 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe8f89-b6ca-4f6d-b80e-1953766b52e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.402001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe8f89-b6ca-4f6d-b80e-1953766b52e2 admin admin] PATH_INFO: `/v3/projects/3ce59d9a516c4ec296e5f4e74627491b/users/4beff9397579496c80fd6e93b7a81542/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.418153 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:44.429866 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe8f89-b6ca-4f6d-b80e-1953766b52e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce59d9a516c4ec296e5f4e74627491b, user_id=4beff9397579496c80fd6e93b7a81542, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:44.433805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-3ade1ff5-5ec7-4f07-a052-89671a096b2e None None] MFA Rules not processed for user `4612504f393546dfbea12ffb3eed5efe`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:44.434188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe8f89-b6ca-4f6d-b80e-1953766b52e2 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:44.434288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3ade1ff5-5ec7-4f07-a052-89671a096b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:44.504609 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 794/1611] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:44.506153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea396160-8566-4ae4-bc20-4b1f3870c469 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.506462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea396160-8566-4ae4-bc20-4b1f3870c469 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.506726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea396160-8566-4ae4-bc20-4b1f3870c469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.526708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-ea396160-8566-4ae4-bc20-4b1f3870c469 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:44.528111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ea396160-8566-4ae4-bc20-4b1f3870c469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:44.538431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 795/1612] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:44.539167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea396160-8566-4ae4-bc20-4b1f3870c469 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:44.548632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 818/1613] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:44 2022] PUT /identity/v3/projects/3ce59d9a516c4ec296e5f4e74627491b/users/4beff9397579496c80fd6e93b7a81542/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:44.549633 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe8f89-b6ca-4f6d-b80e-1953766b52e2 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:44.551239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea396160-8566-4ae4-bc20-4b1f3870c469 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.624157 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38664191-a3a3-494a-b76d-de5b678ba9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.625622 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38664191-a3a3-494a-b76d-de5b678ba9f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.626160 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38664191-a3a3-494a-b76d-de5b678ba9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.626486 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38664191-a3a3-494a-b76d-de5b678ba9f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.628015 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38664191-a3a3-494a-b76d-de5b678ba9f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:44.630144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe8f89-b6ca-4f6d-b80e-1953766b52e2 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.632228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38664191-a3a3-494a-b76d-de5b678ba9f2 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:44.636714 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13d7984-bc11-45c1-811e-debf7c321258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.637488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d7984-bc11-45c1-811e-debf7c321258 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.637898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d7984-bc11-45c1-811e-debf7c321258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.638267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13d7984-bc11-45c1-811e-debf7c321258 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.639024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d7984-bc11-45c1-811e-debf7c321258 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:44.640760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:44.641124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13d7984-bc11-45c1-811e-debf7c321258 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:44.722701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 796/1614] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:21:44 2022] POST /identity/v3/users => generated 291 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:44.723725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38664191-a3a3-494a-b76d-de5b678ba9f2 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:44.758217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 819/1615] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:21:44 2022] POST /identity/v3/projects => generated 385 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:44.758217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d05a6-c39e-4277-b10c-942825ca114f None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.758217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d05a6-c39e-4277-b10c-942825ca114f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.758217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d05a6-c39e-4277-b10c-942825ca114f None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.758822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 820/1616] 10.222.0.60 () {54 vars in 961 bytes} [Wed Jun 15 10:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:21:44.759683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d57dd7-27a5-4a0d-9c39-6427b41d15c0 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.760125 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d57dd7-27a5-4a0d-9c39-6427b41d15c0 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.760414 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d57dd7-27a5-4a0d-9c39-6427b41d15c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.779009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-20d57dd7-27a5-4a0d-9c39-6427b41d15c0 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:44.780229 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-20d57dd7-27a5-4a0d-9c39-6427b41d15c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:44.785615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38664191-a3a3-494a-b76d-de5b678ba9f2 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.846226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6cb2c-e0e0-4020-a7f9-2c8b0b14d56f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.847190 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6cb2c-e0e0-4020-a7f9-2c8b0b14d56f service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.847190 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6cb2c-e0e0-4020-a7f9-2c8b0b14d56f service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.847503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6cb2c-e0e0-4020-a7f9-2c8b0b14d56f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.849812 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 821/1617] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:44.850781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d57dd7-27a5-4a0d-9c39-6427b41d15c0 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:44.861824 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d57dd7-27a5-4a0d-9c39-6427b41d15c0 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.862071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6cb2c-e0e0-4020-a7f9-2c8b0b14d56f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:44.863903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6cb2c-e0e0-4020-a7f9-2c8b0b14d56f service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:44.868595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3c18c-9259-4e12-853b-4b38d2fff241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.869563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3c18c-9259-4e12-853b-4b38d2fff241 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.870122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3c18c-9259-4e12-853b-4b38d2fff241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.870609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc3c18c-9259-4e12-853b-4b38d2fff241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.876922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 797/1618] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:44.877950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6cb2c-e0e0-4020-a7f9-2c8b0b14d56f service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:44.887752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3c18c-9259-4e12-853b-4b38d2fff241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:44.887752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc3c18c-9259-4e12-853b-4b38d2fff241 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:44.889771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6cb2c-e0e0-4020-a7f9-2c8b0b14d56f service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.896247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9ea74-fe1a-4760-a162-4981416472c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.897103 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9ea74-fe1a-4760-a162-4981416472c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.897459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9ea74-fe1a-4760-a162-4981416472c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.897814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9ea74-fe1a-4760-a162-4981416472c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.898638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9ea74-fe1a-4760-a162-4981416472c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:44.900665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9ea74-fe1a-4760-a162-4981416472c5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:44.902143 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 822/1619] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:44.902724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3c18c-9259-4e12-853b-4b38d2fff241 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:44.915560 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc3c18c-9259-4e12-853b-4b38d2fff241 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:44.918378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 798/1620] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:44.920373 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4519a8-f851-4d04-aecc-e7ff348a3fa0 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.920905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4519a8-f851-4d04-aecc-e7ff348a3fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.921772 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4519a8-f851-4d04-aecc-e7ff348a3fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.924907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cacdf6-0496-4621-a74a-486cb9b5ba99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:44.925922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cacdf6-0496-4621-a74a-486cb9b5ba99 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:44.926397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cacdf6-0496-4621-a74a-486cb9b5ba99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:44.926704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cacdf6-0496-4621-a74a-486cb9b5ba99 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:44.927536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cacdf6-0496-4621-a74a-486cb9b5ba99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:44.929778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cacdf6-0496-4621-a74a-486cb9b5ba99 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:44.944468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-3d4519a8-f851-4d04-aecc-e7ff348a3fa0 None None] MFA Rules not processed for user `26f23f8595fc43ad94aee7d0192164a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:44.946076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3d4519a8-f851-4d04-aecc-e7ff348a3fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:45.039212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 823/1621] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:21:44 2022] POST /identity/v3/users => generated 299 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:45.040384 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cacdf6-0496-4621-a74a-486cb9b5ba99 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:45.041928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 799/1622] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Jun 15 10:21:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:45.043150 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4519a8-f851-4d04-aecc-e7ff348a3fa0 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:45.056493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4519a8-f851-4d04-aecc-e7ff348a3fa0 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:45.064684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cacdf6-0496-4621-a74a-486cb9b5ba99 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:45.065450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9912926-41d8-495f-a9dc-c92eff82b30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:45.067300 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9912926-41d8-495f-a9dc-c92eff82b30d admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.067572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9912926-41d8-495f-a9dc-c92eff82b30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.068394 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9912926-41d8-495f-a9dc-c92eff82b30d admin admin] PATH_INFO: `/v3/projects/1220d5ef84ff490bb6afa6eb30d67537/users/b2009af09e2541b0a0f01760c2b94ed8/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.085553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c8892-e860-4831-8eac-6fafa7ae8e32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:45.087602 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9912926-41d8-495f-a9dc-c92eff82b30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1220d5ef84ff490bb6afa6eb30d67537, user_id=b2009af09e2541b0a0f01760c2b94ed8, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:45.088946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c8892-e860-4831-8eac-6fafa7ae8e32 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.089836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c8892-e860-4831-8eac-6fafa7ae8e32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.090690 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c8892-e860-4831-8eac-6fafa7ae8e32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.091219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9912926-41d8-495f-a9dc-c92eff82b30d admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:45.110277 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c8892-e860-4831-8eac-6fafa7ae8e32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:45.114669 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c8892-e860-4831-8eac-6fafa7ae8e32 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:45.156106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 824/1623] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:45.157547 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c8892-e860-4831-8eac-6fafa7ae8e32 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:45.181433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c8892-e860-4831-8eac-6fafa7ae8e32 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:45.194260 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c8e0f-2470-450c-a87d-be6ab6df2beb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:45.195381 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c8e0f-2470-450c-a87d-be6ab6df2beb service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.195949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c8e0f-2470-450c-a87d-be6ab6df2beb service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.196463 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c8e0f-2470-450c-a87d-be6ab6df2beb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.230559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 800/1624] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:45 2022] PUT /identity/v3/projects/1220d5ef84ff490bb6afa6eb30d67537/users/b2009af09e2541b0a0f01760c2b94ed8/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:45.230559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9912926-41d8-495f-a9dc-c92eff82b30d admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:45.246895 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:45.254946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:45.303939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c8e0f-2470-450c-a87d-be6ab6df2beb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:45.305786 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c8e0f-2470-450c-a87d-be6ab6df2beb service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:45.317429 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 825/1625] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:45.321152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5759e99-ad9e-4c88-bd0b-91201f40dca5 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.321152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5759e99-ad9e-4c88-bd0b-91201f40dca5 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.321152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5759e99-ad9e-4c88-bd0b-91201f40dca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.368148 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9912926-41d8-495f-a9dc-c92eff82b30d admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:45.371541 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-b5759e99-ad9e-4c88-bd0b-91201f40dca5 None None] MFA Rules not processed for user `3f8028df66d44c478268a84167c9ff19`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:45.372938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b5759e99-ad9e-4c88-bd0b-91201f40dca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:45.394542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3286d-606d-4fad-8b19-adaf411846e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:45.395290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3286d-606d-4fad-8b19-adaf411846e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.395552 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3286d-606d-4fad-8b19-adaf411846e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.395838 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3286d-606d-4fad-8b19-adaf411846e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.396594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3286d-606d-4fad-8b19-adaf411846e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:45.403868 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3286d-606d-4fad-8b19-adaf411846e5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:45.431647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 801/1626] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:45.431647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3286d-606d-4fad-8b19-adaf411846e5 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:45.459869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3286d-606d-4fad-8b19-adaf411846e5 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:45.469556 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910a633-697a-4dd5-8a03-118130e6b7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:45.470441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910a633-697a-4dd5-8a03-118130e6b7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.470739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910a633-697a-4dd5-8a03-118130e6b7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.472567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910a633-697a-4dd5-8a03-118130e6b7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.472567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910a633-697a-4dd5-8a03-118130e6b7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:45.473967 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910a633-697a-4dd5-8a03-118130e6b7d9 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:45.499680 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 802/1627] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:45.499680 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910a633-697a-4dd5-8a03-118130e6b7d9 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:45.527355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 826/1628] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:45.529833 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5759e99-ad9e-4c88-bd0b-91201f40dca5 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:45.533962 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910a633-697a-4dd5-8a03-118130e6b7d9 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:45.553279 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8727f-3d7b-4db6-8a8b-e7452b792d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:45.554231 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8727f-3d7b-4db6-8a8b-e7452b792d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.554563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8727f-3d7b-4db6-8a8b-e7452b792d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.554809 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8727f-3d7b-4db6-8a8b-e7452b792d59 admin admin] PATH_INFO: `/v3/projects/11d3f24705454698b1916cd621767899/users/566027d44d9d48b6adff9ca1376c1c50/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.557688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5759e99-ad9e-4c88-bd0b-91201f40dca5 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:45.569202 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3d409a-66a8-417a-92b2-06027cc2b43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:45.574718 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d409a-66a8-417a-92b2-06027cc2b43f admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.575014 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d409a-66a8-417a-92b2-06027cc2b43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.577042 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3d409a-66a8-417a-92b2-06027cc2b43f admin admin] PATH_INFO: `/v3/projects/1220d5ef84ff490bb6afa6eb30d67537/users/b2009af09e2541b0a0f01760c2b94ed8/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.579135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8727f-3d7b-4db6-8a8b-e7452b792d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d3f24705454698b1916cd621767899, user_id=566027d44d9d48b6adff9ca1376c1c50, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:45.582786 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d409a-66a8-417a-92b2-06027cc2b43f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1220d5ef84ff490bb6afa6eb30d67537, user_id=b2009af09e2541b0a0f01760c2b94ed8, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:45.583275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8727f-3d7b-4db6-8a8b-e7452b792d59 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:45.587157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3d409a-66a8-417a-92b2-06027cc2b43f admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:45.623180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 827/1629] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:45 2022] PUT /identity/v3/projects/1220d5ef84ff490bb6afa6eb30d67537/users/b2009af09e2541b0a0f01760c2b94ed8/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:45.625522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d1072-15b4-477d-89b0-3f073b63718a None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.626736 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d1072-15b4-477d-89b0-3f073b63718a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.627216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d1072-15b4-477d-89b0-3f073b63718a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.652491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-8f5d1072-15b4-477d-89b0-3f073b63718a None None] MFA Rules not processed for user `3f8028df66d44c478268a84167c9ff19`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:45.654103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-8f5d1072-15b4-477d-89b0-3f073b63718a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:45.658969 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 803/1630] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:45 2022] PUT /identity/v3/projects/11d3f24705454698b1916cd621767899/users/566027d44d9d48b6adff9ca1376c1c50/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:45.661416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bb601-2dc1-42f6-844b-671490bdfa05 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.661957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bb601-2dc1-42f6-844b-671490bdfa05 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.662504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bb601-2dc1-42f6-844b-671490bdfa05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.719697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a89bb601-2dc1-42f6-844b-671490bdfa05 None None] MFA Rules not processed for user `b2009af09e2541b0a0f01760c2b94ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:45.722984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a89bb601-2dc1-42f6-844b-671490bdfa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:45.749685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 828/1631] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:45.750757 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d1072-15b4-477d-89b0-3f073b63718a None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:45.854012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 804/1632] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:45 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:45.855699 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d9555-5050-47df-9c1d-53feb8fbb238 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.856346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d9555-5050-47df-9c1d-53feb8fbb238 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.856633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d9555-5050-47df-9c1d-53feb8fbb238 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.858857 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 805/1633] 10.222.0.60 () {54 vars in 912 bytes} [Wed Jun 15 10:21:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:21:45.860394 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66147c8-ae0e-49ec-bbe0-426c67d8c00c None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.860762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66147c8-ae0e-49ec-bbe0-426c67d8c00c None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.861188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66147c8-ae0e-49ec-bbe0-426c67d8c00c None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.863460 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 806/1634] 10.222.0.60 () {54 vars in 961 bytes} [Wed Jun 15 10:21:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:21:45.865795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66147c8-ae0e-49ec-bbe0-426c67d8c00c None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:45.877814 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d1072-15b4-477d-89b0-3f073b63718a None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:45.890723 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066b15b-8891-459c-933f-5fe46d0b9b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:45.894465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066b15b-8891-459c-933f-5fe46d0b9b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.894465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066b15b-8891-459c-933f-5fe46d0b9b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.894685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066b15b-8891-459c-933f-5fe46d0b9b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.896462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066b15b-8891-459c-933f-5fe46d0b9b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:45.901420 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066b15b-8891-459c-933f-5fe46d0b9b90 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:45.931941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 829/1635] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:45.934227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5197f-9857-43ab-ba35-9bbeea872a00 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.934728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5197f-9857-43ab-ba35-9bbeea872a00 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.935221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5197f-9857-43ab-ba35-9bbeea872a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:45.955721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:45.964339 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-56c5197f-9857-43ab-ba35-9bbeea872a00 None None] MFA Rules not processed for user `26f23f8595fc43ad94aee7d0192164a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:45.965844 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-56c5197f-9857-43ab-ba35-9bbeea872a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:45.971602 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66147c8-ae0e-49ec-bbe0-426c67d8c00c None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:45.982270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0f04c-e10c-4f50-bafd-f9ae8a417e6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:45.985936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0f04c-e10c-4f50-bafd-f9ae8a417e6e service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:45.985936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0f04c-e10c-4f50-bafd-f9ae8a417e6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:45.985936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0f04c-e10c-4f50-bafd-f9ae8a417e6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:46.031624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:46.050611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0f04c-e10c-4f50-bafd-f9ae8a417e6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:46.056056 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0f04c-e10c-4f50-bafd-f9ae8a417e6e service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:46.077153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 807/1636] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:45 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:46.079493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f79b87-b600-474d-a041-6cf3ca83ff3d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:46.079827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f79b87-b600-474d-a041-6cf3ca83ff3d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:46.080159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f79b87-b600-474d-a041-6cf3ca83ff3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:46.104042 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 830/1637] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Jun 15 10:21:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:46.104042 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5197f-9857-43ab-ba35-9bbeea872a00 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:46.111574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5197f-9857-43ab-ba35-9bbeea872a00 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:46.118434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb19bc5-e3d4-44e2-80a5-efa1730396f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:46.119356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb19bc5-e3d4-44e2-80a5-efa1730396f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:46.119683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb19bc5-e3d4-44e2-80a5-efa1730396f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:46.119975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb19bc5-e3d4-44e2-80a5-efa1730396f7 admin admin] PATH_INFO: `/v3/projects/11d3f24705454698b1916cd621767899/users/566027d44d9d48b6adff9ca1376c1c50/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:46.120175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-78f79b87-b600-474d-a041-6cf3ca83ff3d None None] MFA Rules not processed for user `26f23f8595fc43ad94aee7d0192164a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:46.123140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb19bc5-e3d4-44e2-80a5-efa1730396f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d3f24705454698b1916cd621767899, user_id=566027d44d9d48b6adff9ca1376c1c50, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:46.123314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-78f79b87-b600-474d-a041-6cf3ca83ff3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:46.127716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb19bc5-e3d4-44e2-80a5-efa1730396f7 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:46.149829 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 831/1638] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:46 2022] PUT /identity/v3/projects/11d3f24705454698b1916cd621767899/users/566027d44d9d48b6adff9ca1376c1c50/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:46.152031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb19bc5-e3d4-44e2-80a5-efa1730396f7 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:46.180172 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 808/1639] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jun 15 10:21:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:46.182700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0af92c-3b8e-4114-b52b-1283dce2bf86 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:46.189616 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0af92c-3b8e-4114-b52b-1283dce2bf86 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:46.190128 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0af92c-3b8e-4114-b52b-1283dce2bf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:46.261441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-9a0af92c-3b8e-4114-b52b-1283dce2bf86 None None] MFA Rules not processed for user `566027d44d9d48b6adff9ca1376c1c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:46.264093 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9a0af92c-3b8e-4114-b52b-1283dce2bf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:46.264723 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb19bc5-e3d4-44e2-80a5-efa1730396f7 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:46.272667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60ef04-a903-40ad-9315-cf0636d0639b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:46.273557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60ef04-a903-40ad-9315-cf0636d0639b service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:46.273886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60ef04-a903-40ad-9315-cf0636d0639b service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:46.274177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60ef04-a903-40ad-9315-cf0636d0639b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:46.392949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60ef04-a903-40ad-9315-cf0636d0639b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:46.395855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60ef04-a903-40ad-9315-cf0636d0639b service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:46.419924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 832/1640] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:46.421171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60ef04-a903-40ad-9315-cf0636d0639b service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:46.478455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 809/1641] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:46.499972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60ef04-a903-40ad-9315-cf0636d0639b service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:46.506335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc4262-4706-4c5a-a2e9-d7b507893bdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:46.507631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc4262-4706-4c5a-a2e9-d7b507893bdf service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:46.508187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc4262-4706-4c5a-a2e9-d7b507893bdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:46.508718 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc4262-4706-4c5a-a2e9-d7b507893bdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:46.530571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc4262-4706-4c5a-a2e9-d7b507893bdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:46.533599 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc4262-4706-4c5a-a2e9-d7b507893bdf service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:46.567699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 833/1642] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:46.575259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0af92c-3b8e-4114-b52b-1283dce2bf86 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:46.608401 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0af92c-3b8e-4114-b52b-1283dce2bf86 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:46.622206 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b44539a-92d0-4b9d-9572-6d78fb797c3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:46.623268 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44539a-92d0-4b9d-9572-6d78fb797c3f service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:46.623770 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44539a-92d0-4b9d-9572-6d78fb797c3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:46.624271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b44539a-92d0-4b9d-9572-6d78fb797c3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:46.659109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44539a-92d0-4b9d-9572-6d78fb797c3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:46.663262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b44539a-92d0-4b9d-9572-6d78fb797c3f service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:46.719591 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 810/1643] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:46.865823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ac223-e227-43c2-8e68-88affca6b7a6 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:46.866159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ac223-e227-43c2-8e68-88affca6b7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:46.866415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ac223-e227-43c2-8e68-88affca6b7a6 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:46.868881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 834/1644] 10.222.0.60 () {54 vars in 912 bytes} [Wed Jun 15 10:21:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:21:46.877166 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ca4e6-e30b-4305-8b33-e4a8161ca1bc None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:46.877519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ca4e6-e30b-4305-8b33-e4a8161ca1bc None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:46.877802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ca4e6-e30b-4305-8b33-e4a8161ca1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:46.903334 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-9a5ca4e6-e30b-4305-8b33-e4a8161ca1bc None None] MFA Rules not processed for user `26f23f8595fc43ad94aee7d0192164a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:46.904742 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9a5ca4e6-e30b-4305-8b33-e4a8161ca1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:46.925717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 811/1645] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jun 15 10:21:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:21:46.953765 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ac223-e227-43c2-8e68-88affca6b7a6 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:46.963985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c85ac223-e227-43c2-8e68-88affca6b7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:47.024311 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ca4e6-e30b-4305-8b33-e4a8161ca1bc None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:47.085596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:47.199976 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ac223-e227-43c2-8e68-88affca6b7a6 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:47.202269 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ca4e6-e30b-4305-8b33-e4a8161ca1bc None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:47.213884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2450d453-9090-4fcf-828a-bd9705983838 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fbf33189aa94f4f973d558d615a23e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:47.214568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450d453-9090-4fcf-828a-bd9705983838 service glance] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:47.215908 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450d453-9090-4fcf-828a-bd9705983838 service glance] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:47.216248 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2450d453-9090-4fcf-828a-bd9705983838 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:47.223133 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea57a84-2f58-47af-bbb4-bfc217dc2b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:47.224149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea57a84-2f58-47af-bbb4-bfc217dc2b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:47.224444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea57a84-2f58-47af-bbb4-bfc217dc2b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:47.224728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea57a84-2f58-47af-bbb4-bfc217dc2b84 admin admin] PATH_INFO: `/v3/users/7e2d196819c54e049858fc19139a0bda` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:47.226743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea57a84-2f58-47af-bbb4-bfc217dc2b84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e2d196819c54e049858fc19139a0bda)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:47.227801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: WARNING py.warnings [None req-8ea57a84-2f58-47af-bbb4-bfc217dc2b84 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:21:47.227801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: warnings.warn(msg) Jun 15 10:21:47.227801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: Jun 15 10:21:47.229222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea57a84-2f58-47af-bbb4-bfc217dc2b84 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:47.236934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450d453-9090-4fcf-828a-bd9705983838 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:47.239885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2450d453-9090-4fcf-828a-bd9705983838 service glance] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:47.263563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 835/1646] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:47.549017 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8ea57a84-2f58-47af-bbb4-bfc217dc2b84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e2d196819c54e049858fc19139a0bda'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:47.674389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8ea57a84-2f58-47af-bbb4-bfc217dc2b84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e2d196819c54e049858fc19139a0bda'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:47.676707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8ea57a84-2f58-47af-bbb4-bfc217dc2b84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e2d196819c54e049858fc19139a0bda'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:47.681792 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8ea57a84-2f58-47af-bbb4-bfc217dc2b84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e2d196819c54e049858fc19139a0bda'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:47.693177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:47.765612 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8ea57a84-2f58-47af-bbb4-bfc217dc2b84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e2d196819c54e049858fc19139a0bda'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:47.796501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 812/1647] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:21:47 2022] DELETE /identity/v3/users/7e2d196819c54e049858fc19139a0bda => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:47.801133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a516394-1da3-4c5a-9fd8-978317edfebf None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:47.801426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a516394-1da3-4c5a-9fd8-978317edfebf None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:47.801686 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a516394-1da3-4c5a-9fd8-978317edfebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:47.839068 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-0a516394-1da3-4c5a-9fd8-978317edfebf None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:47.842267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0a516394-1da3-4c5a-9fd8-978317edfebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:47.879379 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aca495-d4f2-4318-88d0-b6d6ad3341e9 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:47.879379 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aca495-d4f2-4318-88d0-b6d6ad3341e9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:47.879656 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aca495-d4f2-4318-88d0-b6d6ad3341e9 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:47.881444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 813/1648] 10.222.0.60 () {54 vars in 912 bytes} [Wed Jun 15 10:21:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:21:47.888093 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba137c-6295-4eb2-ac2b-388b203a9bc1 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:47.888390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba137c-6295-4eb2-ac2b-388b203a9bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:47.890502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba137c-6295-4eb2-ac2b-388b203a9bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:47.981748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 836/1649] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:48.006096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-38ba137c-6295-4eb2-ac2b-388b203a9bc1 None None] MFA Rules not processed for user `ae926ce83d464c768baddd90de31aa40`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:48.007364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-38ba137c-6295-4eb2-ac2b-388b203a9bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:48.076982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a516394-1da3-4c5a-9fd8-978317edfebf None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:48.079314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0a516394-1da3-4c5a-9fd8-978317edfebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:48.194805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 814/1650] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jun 15 10:21:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:48.246144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:48.248701 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a516394-1da3-4c5a-9fd8-978317edfebf None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:48.260388 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bcb51d-6275-4d93-bcb4-db2145ae143e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:48.261214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bcb51d-6275-4d93-bcb4-db2145ae143e service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:48.261440 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bcb51d-6275-4d93-bcb4-db2145ae143e service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:48.460892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bcb51d-6275-4d93-bcb4-db2145ae143e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:48.460892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bcb51d-6275-4d93-bcb4-db2145ae143e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:48.460892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bcb51d-6275-4d93-bcb4-db2145ae143e service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:48.460892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 837/1651] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:48.499073 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba137c-6295-4eb2-ac2b-388b203a9bc1 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:48.499073 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-38ba137c-6295-4eb2-ac2b-388b203a9bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:48.499073 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba137c-6295-4eb2-ac2b-388b203a9bc1 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:48.502583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbeb996-0304-431f-80e1-6620e678f4e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:48.502583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbeb996-0304-431f-80e1-6620e678f4e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:48.502583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbeb996-0304-431f-80e1-6620e678f4e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:48.502583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbeb996-0304-431f-80e1-6620e678f4e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:48.504429 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbeb996-0304-431f-80e1-6620e678f4e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:48.504429 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbeb996-0304-431f-80e1-6620e678f4e5 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:48.504429 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 815/1652] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:49.766803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c724324-015c-4d84-8e4f-32cd7ecd7203 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:49.767878 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c724324-015c-4d84-8e4f-32cd7ecd7203 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:49.768310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c724324-015c-4d84-8e4f-32cd7ecd7203 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:49.771287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 838/1653] 10.222.0.60 () {54 vars in 960 bytes} [Wed Jun 15 10:21:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:21:49.780125 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1e126-f1ec-404a-a898-dbed1eb0ae3e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:49.784917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1e126-f1ec-404a-a898-dbed1eb0ae3e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:49.784917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1e126-f1ec-404a-a898-dbed1eb0ae3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:49.820839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-19c1e126-f1ec-404a-a898-dbed1eb0ae3e None None] MFA Rules not processed for user `12899a7a59b54e3199b277ff94e30d42`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:49.822467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-19c1e126-f1ec-404a-a898-dbed1eb0ae3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:49.928682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 816/1654] 10.222.0.60 () {58 vars in 1057 bytes} [Wed Jun 15 10:21:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:49.936215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c724324-015c-4d84-8e4f-32cd7ecd7203 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:49.953021 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c724324-015c-4d84-8e4f-32cd7ecd7203 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:49.960508 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a37eb9-5d43-4559-a18e-11da32ca9c3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:49.961610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a37eb9-5d43-4559-a18e-11da32ca9c3c service placement] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:49.962099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a37eb9-5d43-4559-a18e-11da32ca9c3c service placement] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:49.962602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a37eb9-5d43-4559-a18e-11da32ca9c3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:49.965480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c7a37eb9-5d43-4559-a18e-11da32ca9c3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:49.981611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a37eb9-5d43-4559-a18e-11da32ca9c3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:49.983683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a37eb9-5d43-4559-a18e-11da32ca9c3c service placement] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:49.998309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 839/1655] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:50.051088 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1e126-f1ec-404a-a898-dbed1eb0ae3e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:50.056503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-19c1e126-f1ec-404a-a898-dbed1eb0ae3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:50.102400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1e126-f1ec-404a-a898-dbed1eb0ae3e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:50.115201 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3e430-bdde-4411-bba0-9cba8a87c0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:50.116157 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3e430-bdde-4411-bba0-9cba8a87c0ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:50.116449 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3e430-bdde-4411-bba0-9cba8a87c0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:50.118485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3e430-bdde-4411-bba0-9cba8a87c0ea admin admin] PATH_INFO: `/v3/projects/b308be72b7e440058d72246650648cf1` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:50.123815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3e430-bdde-4411-bba0-9cba8a87c0ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=b308be72b7e440058d72246650648cf1)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:50.131305 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3e430-bdde-4411-bba0-9cba8a87c0ea admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:50.306176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:50.308089 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-07f3e430-bdde-4411-bba0-9cba8a87c0ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b308be72b7e440058d72246650648cf1', 'request_id': 'req-07f3e430-bdde-4411-bba0-9cba8a87c0ea'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:50.318509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-07f3e430-bdde-4411-bba0-9cba8a87c0ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b308be72b7e440058d72246650648cf1', 'request_id': 'req-07f3e430-bdde-4411-bba0-9cba8a87c0ea'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:50.440273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-07f3e430-bdde-4411-bba0-9cba8a87c0ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b308be72b7e440058d72246650648cf1', 'request_id': 'req-07f3e430-bdde-4411-bba0-9cba8a87c0ea'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:50.503880 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-07f3e430-bdde-4411-bba0-9cba8a87c0ea admin admin] The token cache is being invalidate because project b308be72b7e440058d72246650648cf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:21:50.504410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-07f3e430-bdde-4411-bba0-9cba8a87c0ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:50.508289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-07f3e430-bdde-4411-bba0-9cba8a87c0ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:50.517872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a37eb9-5d43-4559-a18e-11da32ca9c3c service placement] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:50.518368 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 817/1656] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:21:50 2022] DELETE /identity/v3/projects/b308be72b7e440058d72246650648cf1 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:50.539252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a37eb9-5d43-4559-a18e-11da32ca9c3c service placement] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:50.689708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1584868e-9548-4b2b-b47a-c4988efc8ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:50.691011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584868e-9548-4b2b-b47a-c4988efc8ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:50.691640 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584868e-9548-4b2b-b47a-c4988efc8ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:50.692312 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584868e-9548-4b2b-b47a-c4988efc8ed8 admin admin] PATH_INFO: `/v3/users/566027d44d9d48b6adff9ca1376c1c50` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:50.696851 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584868e-9548-4b2b-b47a-c4988efc8ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=566027d44d9d48b6adff9ca1376c1c50)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:50.699662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1584868e-9548-4b2b-b47a-c4988efc8ed8 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:50.996570 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1584868e-9548-4b2b-b47a-c4988efc8ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '566027d44d9d48b6adff9ca1376c1c50'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:51.032740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1584868e-9548-4b2b-b47a-c4988efc8ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '566027d44d9d48b6adff9ca1376c1c50'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:51.151274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1584868e-9548-4b2b-b47a-c4988efc8ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '566027d44d9d48b6adff9ca1376c1c50'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:51.171797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1584868e-9548-4b2b-b47a-c4988efc8ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '566027d44d9d48b6adff9ca1376c1c50'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:51.173092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1584868e-9548-4b2b-b47a-c4988efc8ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '566027d44d9d48b6adff9ca1376c1c50'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:51.188219 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 840/1657] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:21:50 2022] DELETE /identity/v3/users/566027d44d9d48b6adff9ca1376c1c50 => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:21:51.724154 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3e430-bdde-4411-bba0-9cba8a87c0ea admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:51.727520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-07f3e430-bdde-4411-bba0-9cba8a87c0ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:51.773650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3e430-bdde-4411-bba0-9cba8a87c0ea admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:52.066621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695139e-bcac-4cbe-917f-711f5d3f6d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:52.067994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695139e-bcac-4cbe-917f-711f5d3f6d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:52.068665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695139e-bcac-4cbe-917f-711f5d3f6d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:52.068974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695139e-bcac-4cbe-917f-711f5d3f6d22 admin admin] PATH_INFO: `/v3/users/f5ca2c3d2a964f959d50eba45287184c` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:52.073860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695139e-bcac-4cbe-917f-711f5d3f6d22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ca2c3d2a964f959d50eba45287184c)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:52.076101 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695139e-bcac-4cbe-917f-711f5d3f6d22 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:52.350405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c695139e-bcac-4cbe-917f-711f5d3f6d22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ca2c3d2a964f959d50eba45287184c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:52.466775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c695139e-bcac-4cbe-917f-711f5d3f6d22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ca2c3d2a964f959d50eba45287184c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:52.467626 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c695139e-bcac-4cbe-917f-711f5d3f6d22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ca2c3d2a964f959d50eba45287184c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:52.468353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c695139e-bcac-4cbe-917f-711f5d3f6d22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ca2c3d2a964f959d50eba45287184c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:52.554145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c695139e-bcac-4cbe-917f-711f5d3f6d22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ca2c3d2a964f959d50eba45287184c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:52.605397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 818/1658] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:21:51 2022] DELETE /identity/v3/users/f5ca2c3d2a964f959d50eba45287184c => generated 0 bytes in 882 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:21:52.610738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b53fb-5050-42d3-9112-1af3566bb851 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:52.612950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b53fb-5050-42d3-9112-1af3566bb851 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:52.612950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b53fb-5050-42d3-9112-1af3566bb851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:52.668923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-0f8b53fb-5050-42d3-9112-1af3566bb851 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:52.671615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0f8b53fb-5050-42d3-9112-1af3566bb851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:52.791716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:52.819291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 841/1659] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:52.843326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695139e-bcac-4cbe-917f-711f5d3f6d22 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:52.845310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c695139e-bcac-4cbe-917f-711f5d3f6d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:52.900181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b53fb-5050-42d3-9112-1af3566bb851 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:52.901977 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0f8b53fb-5050-42d3-9112-1af3566bb851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:52.922395 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b53fb-5050-42d3-9112-1af3566bb851 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:52.937124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a116e18-9d4e-4699-9953-ec9b7e5925b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:52.939033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a116e18-9d4e-4699-9953-ec9b7e5925b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:52.939033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a116e18-9d4e-4699-9953-ec9b7e5925b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:52.939033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a116e18-9d4e-4699-9953-ec9b7e5925b8 admin admin] PATH_INFO: `/v3/projects/11d3f24705454698b1916cd621767899` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:52.940729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a116e18-9d4e-4699-9953-ec9b7e5925b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11d3f24705454698b1916cd621767899)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:52.942086 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: WARNING py.warnings [None req-5a116e18-9d4e-4699-9953-ec9b7e5925b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 15 10:21:52.942086 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: warnings.warn(msg) Jun 15 10:21:52.942086 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: Jun 15 10:21:52.943296 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a116e18-9d4e-4699-9953-ec9b7e5925b8 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:52.978350 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695139e-bcac-4cbe-917f-711f5d3f6d22 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:52.989682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189e402-763b-46a4-a1f0-3372210d8e88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:52.990495 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e402-763b-46a4-a1f0-3372210d8e88 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:52.990780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e402-763b-46a4-a1f0-3372210d8e88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:52.991572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e402-763b-46a4-a1f0-3372210d8e88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:53.028756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189e402-763b-46a4-a1f0-3372210d8e88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:53.029753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189e402-763b-46a4-a1f0-3372210d8e88 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:53.042478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:53.056588 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 819/1660] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:53.142329 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a116e18-9d4e-4699-9953-ec9b7e5925b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11d3f24705454698b1916cd621767899', 'request_id': 'req-5a116e18-9d4e-4699-9953-ec9b7e5925b8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:53.181456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a116e18-9d4e-4699-9953-ec9b7e5925b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11d3f24705454698b1916cd621767899', 'request_id': 'req-5a116e18-9d4e-4699-9953-ec9b7e5925b8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:53.192118 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a116e18-9d4e-4699-9953-ec9b7e5925b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11d3f24705454698b1916cd621767899', 'request_id': 'req-5a116e18-9d4e-4699-9953-ec9b7e5925b8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:53.328245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a116e18-9d4e-4699-9953-ec9b7e5925b8 admin admin] The token cache is being invalidate because project 11d3f24705454698b1916cd621767899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:21:53.328821 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a116e18-9d4e-4699-9953-ec9b7e5925b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:53.332060 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a116e18-9d4e-4699-9953-ec9b7e5925b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:53.364438 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 842/1661] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:21:52 2022] DELETE /identity/v3/projects/11d3f24705454698b1916cd621767899 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:21:53.369214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b8e37-4b51-426d-8395-a9ab66bbfdfe None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:53.369833 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b8e37-4b51-426d-8395-a9ab66bbfdfe None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:53.370304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b8e37-4b51-426d-8395-a9ab66bbfdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:53.406095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-322b8e37-4b51-426d-8395-a9ab66bbfdfe None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:53.407747 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-322b8e37-4b51-426d-8395-a9ab66bbfdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:53.565309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 820/1662] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:21:53.574458 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d3d67-8fe6-46de-9f42-b34bce23f7ea None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:53.574594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d3d67-8fe6-46de-9f42-b34bce23f7ea None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:53.574918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d3d67-8fe6-46de-9f42-b34bce23f7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:53.615270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-4c1d3d67-8fe6-46de-9f42-b34bce23f7ea None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:53.637160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-4c1d3d67-8fe6-46de-9f42-b34bce23f7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:53.663501 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 843/1663] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:53.668779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b8e37-4b51-426d-8395-a9ab66bbfdfe None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:53.682063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b8e37-4b51-426d-8395-a9ab66bbfdfe None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:53.691452 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835bc15-a7e3-48c3-a1da-e95abe69d390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:53.692991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835bc15-a7e3-48c3-a1da-e95abe69d390 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:53.693498 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835bc15-a7e3-48c3-a1da-e95abe69d390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:53.693953 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835bc15-a7e3-48c3-a1da-e95abe69d390 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:53.694988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835bc15-a7e3-48c3-a1da-e95abe69d390 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:53.695515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835bc15-a7e3-48c3-a1da-e95abe69d390 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:53.697418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835bc15-a7e3-48c3-a1da-e95abe69d390 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:53.742306 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d3d67-8fe6-46de-9f42-b34bce23f7ea None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:53.759216 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 821/1664] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:53.763066 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d3d67-8fe6-46de-9f42-b34bce23f7ea None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:53.770540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc7add-73cc-4eeb-b46d-1b1b7ecf3635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:53.772129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc7add-73cc-4eeb-b46d-1b1b7ecf3635 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:53.772631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc7add-73cc-4eeb-b46d-1b1b7ecf3635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:53.773156 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc7add-73cc-4eeb-b46d-1b1b7ecf3635 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:53.774145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc7add-73cc-4eeb-b46d-1b1b7ecf3635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:53.776420 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc7add-73cc-4eeb-b46d-1b1b7ecf3635 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:53.910699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 844/1665] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:21:53 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:53.915756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835bc15-a7e3-48c3-a1da-e95abe69d390 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:53.932584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835bc15-a7e3-48c3-a1da-e95abe69d390 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:54.146655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ab01d-8798-4e55-870b-a11484b6c7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:54.146655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ab01d-8798-4e55-870b-a11484b6c7ff admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:54.146655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ab01d-8798-4e55-870b-a11484b6c7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:54.146917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ab01d-8798-4e55-870b-a11484b6c7ff admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:54.147695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ab01d-8798-4e55-870b-a11484b6c7ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:54.150457 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ab01d-8798-4e55-870b-a11484b6c7ff admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:54.230902 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc7add-73cc-4eeb-b46d-1b1b7ecf3635 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:54.233544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2ebc7add-73cc-4eeb-b46d-1b1b7ecf3635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:54.272058 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 822/1666] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:21:53 2022] POST /identity/v3/users => generated 292 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:54.273726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ab01d-8798-4e55-870b-a11484b6c7ff admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:54.289930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc7add-73cc-4eeb-b46d-1b1b7ecf3635 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:54.297816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ab01d-8798-4e55-870b-a11484b6c7ff admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:54.308094 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0385e8-644a-41dd-b130-6e7900400ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:54.313545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0385e8-644a-41dd-b130-6e7900400ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:54.313869 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0385e8-644a-41dd-b130-6e7900400ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:54.314218 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0385e8-644a-41dd-b130-6e7900400ec9 admin admin] PATH_INFO: `/v3/projects/8e28d54c66fd43b7bd68bd5a95f5df4d` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:54.320982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0385e8-644a-41dd-b130-6e7900400ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e28d54c66fd43b7bd68bd5a95f5df4d)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:54.322975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0385e8-644a-41dd-b130-6e7900400ec9 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:54.325497 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544ab6d-0675-4d6f-8904-bd9dfd10a773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:54.329323 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544ab6d-0675-4d6f-8904-bd9dfd10a773 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:54.329617 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544ab6d-0675-4d6f-8904-bd9dfd10a773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:54.329893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544ab6d-0675-4d6f-8904-bd9dfd10a773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:54.331336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544ab6d-0675-4d6f-8904-bd9dfd10a773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:54.337962 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544ab6d-0675-4d6f-8904-bd9dfd10a773 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:54.418406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 823/1667] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:54.419508 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544ab6d-0675-4d6f-8904-bd9dfd10a773 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:54.467922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544ab6d-0675-4d6f-8904-bd9dfd10a773 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:54.478186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb517b89-1c3b-4c78-ac3a-8ad02315cf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:54.479973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb517b89-1c3b-4c78-ac3a-8ad02315cf81 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:54.480278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb517b89-1c3b-4c78-ac3a-8ad02315cf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:54.481343 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb517b89-1c3b-4c78-ac3a-8ad02315cf81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:54.483214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb517b89-1c3b-4c78-ac3a-8ad02315cf81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:54.484145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb517b89-1c3b-4c78-ac3a-8ad02315cf81 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:54.503697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 824/1668] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:54.508547 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb517b89-1c3b-4c78-ac3a-8ad02315cf81 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:54.534827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb517b89-1c3b-4c78-ac3a-8ad02315cf81 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:54.561298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46acfd60-b1fc-4ab0-ba5a-d144d8d427f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:54.563987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acfd60-b1fc-4ab0-ba5a-d144d8d427f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:54.564357 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acfd60-b1fc-4ab0-ba5a-d144d8d427f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:54.564655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46acfd60-b1fc-4ab0-ba5a-d144d8d427f5 admin admin] PATH_INFO: `/v3/projects/7e4f842389d840738362f2594c3437f7/users/88adc6706e4b4c339f14786c9ff24b68/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:54.587216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2e0385e8-644a-41dd-b130-6e7900400ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e28d54c66fd43b7bd68bd5a95f5df4d', 'request_id': 'req-2e0385e8-644a-41dd-b130-6e7900400ec9'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:54.598396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2e0385e8-644a-41dd-b130-6e7900400ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e28d54c66fd43b7bd68bd5a95f5df4d', 'request_id': 'req-2e0385e8-644a-41dd-b130-6e7900400ec9'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:54.610512 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46acfd60-b1fc-4ab0-ba5a-d144d8d427f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e4f842389d840738362f2594c3437f7, user_id=88adc6706e4b4c339f14786c9ff24b68, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:54.618178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2e0385e8-644a-41dd-b130-6e7900400ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e28d54c66fd43b7bd68bd5a95f5df4d', 'request_id': 'req-2e0385e8-644a-41dd-b130-6e7900400ec9'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:54.622122 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46acfd60-b1fc-4ab0-ba5a-d144d8d427f5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:54.729360 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2e0385e8-644a-41dd-b130-6e7900400ec9 admin admin] The token cache is being invalidate because project 8e28d54c66fd43b7bd68bd5a95f5df4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:21:54.730542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 825/1669] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:54 2022] PUT /identity/v3/projects/7e4f842389d840738362f2594c3437f7/users/88adc6706e4b4c339f14786c9ff24b68/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:54.730755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2e0385e8-644a-41dd-b130-6e7900400ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:54.731891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2e0385e8-644a-41dd-b130-6e7900400ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:54.733693 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 845/1670] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:21:54 2022] DELETE /identity/v3/projects/8e28d54c66fd43b7bd68bd5a95f5df4d => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:54.744353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03238d-da2e-4e0f-9d79-785a30862683 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:54.745504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03238d-da2e-4e0f-9d79-785a30862683 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:54.745916 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0385e8-644a-41dd-b130-6e7900400ec9 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:54.746272 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03238d-da2e-4e0f-9d79-785a30862683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:54.753946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2e0385e8-644a-41dd-b130-6e7900400ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:54.822510 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-de03238d-da2e-4e0f-9d79-785a30862683 None None] MFA Rules not processed for user `88adc6706e4b4c339f14786c9ff24b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:54.823867 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-de03238d-da2e-4e0f-9d79-785a30862683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:54.865286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:54.865869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:54.951009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0385e8-644a-41dd-b130-6e7900400ec9 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:54.963922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 826/1671] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:54.965190 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03238d-da2e-4e0f-9d79-785a30862683 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:54.966656 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-de03238d-da2e-4e0f-9d79-785a30862683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:55.067711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03238d-da2e-4e0f-9d79-785a30862683 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:55.073163 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25d2e4-ff2c-4405-84d6-3c2a0831d0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:55.074348 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25d2e4-ff2c-4405-84d6-3c2a0831d0c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:55.074780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25d2e4-ff2c-4405-84d6-3c2a0831d0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:55.076358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25d2e4-ff2c-4405-84d6-3c2a0831d0c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:55.076358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25d2e4-ff2c-4405-84d6-3c2a0831d0c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:55.086234 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25d2e4-ff2c-4405-84d6-3c2a0831d0c1 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:55.107259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cda050-c9fd-4a10-b052-3bd0c5d6b452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:55.110546 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cda050-c9fd-4a10-b052-3bd0c5d6b452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:55.110546 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cda050-c9fd-4a10-b052-3bd0c5d6b452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:55.110546 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cda050-c9fd-4a10-b052-3bd0c5d6b452 admin admin] PATH_INFO: `/v3/users/b2009af09e2541b0a0f01760c2b94ed8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:55.111657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cda050-c9fd-4a10-b052-3bd0c5d6b452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2009af09e2541b0a0f01760c2b94ed8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:55.114113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cda050-c9fd-4a10-b052-3bd0c5d6b452 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:55.205070 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 827/1672] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:21:54 2022] POST /identity/v3/projects => generated 387 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:55.207369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25d2e4-ff2c-4405-84d6-3c2a0831d0c1 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:55.345200 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-83cda050-c9fd-4a10-b052-3bd0c5d6b452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2009af09e2541b0a0f01760c2b94ed8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:55.362313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25d2e4-ff2c-4405-84d6-3c2a0831d0c1 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:55.367986 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca30c1-0702-472b-9ab8-9f39727ecaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:55.368840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca30c1-0702-472b-9ab8-9f39727ecaf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:55.369231 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca30c1-0702-472b-9ab8-9f39727ecaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:55.369589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca30c1-0702-472b-9ab8-9f39727ecaf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:55.370901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca30c1-0702-472b-9ab8-9f39727ecaf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:55.373878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca30c1-0702-472b-9ab8-9f39727ecaf5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:55.374188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-83cda050-c9fd-4a10-b052-3bd0c5d6b452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2009af09e2541b0a0f01760c2b94ed8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:55.479921 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-83cda050-c9fd-4a10-b052-3bd0c5d6b452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2009af09e2541b0a0f01760c2b94ed8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:55.495352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-83cda050-c9fd-4a10-b052-3bd0c5d6b452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2009af09e2541b0a0f01760c2b94ed8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:55.499174 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-83cda050-c9fd-4a10-b052-3bd0c5d6b452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2009af09e2541b0a0f01760c2b94ed8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:55.528229 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 846/1673] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:21:54 2022] DELETE /identity/v3/users/b2009af09e2541b0a0f01760c2b94ed8 => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:55.549403 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 828/1674] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:21:55 2022] POST /identity/v3/users => generated 294 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:55.554459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cda050-c9fd-4a10-b052-3bd0c5d6b452 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:55.556119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-83cda050-c9fd-4a10-b052-3bd0c5d6b452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:55.779526 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cda050-c9fd-4a10-b052-3bd0c5d6b452 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:55.791472 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1687e3-8342-42fa-bb53-070aa3eb9dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:55.793657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1687e3-8342-42fa-bb53-070aa3eb9dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:55.793657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1687e3-8342-42fa-bb53-070aa3eb9dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:55.793657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1687e3-8342-42fa-bb53-070aa3eb9dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:55.794694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1687e3-8342-42fa-bb53-070aa3eb9dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:55.804010 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1687e3-8342-42fa-bb53-070aa3eb9dc2 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:55.827623 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 847/1675] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:55.828492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca30c1-0702-472b-9ab8-9f39727ecaf5 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:55.844850 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca30c1-0702-472b-9ab8-9f39727ecaf5 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:55.854430 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7000432-f92d-46fe-9950-df6ff31a6993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:55.855021 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7000432-f92d-46fe-9950-df6ff31a6993 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:55.855338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7000432-f92d-46fe-9950-df6ff31a6993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:55.855598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7000432-f92d-46fe-9950-df6ff31a6993 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:55.856365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7000432-f92d-46fe-9950-df6ff31a6993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:55.858059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7000432-f92d-46fe-9950-df6ff31a6993 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:55.892762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 829/1676] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:21:55.898361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1687e3-8342-42fa-bb53-070aa3eb9dc2 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:55.921614 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1687e3-8342-42fa-bb53-070aa3eb9dc2 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:55.934830 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac274c6f-a801-4e21-bee2-63eb6bb6bbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:55.937228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac274c6f-a801-4e21-bee2-63eb6bb6bbcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:55.937659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac274c6f-a801-4e21-bee2-63eb6bb6bbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:55.938047 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac274c6f-a801-4e21-bee2-63eb6bb6bbcc admin admin] PATH_INFO: `/v3/projects/3640e95659274ba3ad199bf69330750e/users/4c24d333f23d4718b9b954bf470b9871/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:55.984006 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac274c6f-a801-4e21-bee2-63eb6bb6bbcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3640e95659274ba3ad199bf69330750e, user_id=4c24d333f23d4718b9b954bf470b9871, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:55.996939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac274c6f-a801-4e21-bee2-63eb6bb6bbcc admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:56.121602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:56.157252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 848/1677] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:55 2022] PUT /identity/v3/projects/3640e95659274ba3ad199bf69330750e/users/4c24d333f23d4718b9b954bf470b9871/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:56.169545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df392a1e-0a93-4fac-a4ed-d6d65c89f272 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:56.170083 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df392a1e-0a93-4fac-a4ed-d6d65c89f272 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:56.170083 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df392a1e-0a93-4fac-a4ed-d6d65c89f272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:56.250133 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-df392a1e-0a93-4fac-a4ed-d6d65c89f272 None None] MFA Rules not processed for user `4c24d333f23d4718b9b954bf470b9871`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:56.254686 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-df392a1e-0a93-4fac-a4ed-d6d65c89f272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:56.282053 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:56.539950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 830/1678] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:56.540723 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac274c6f-a801-4e21-bee2-63eb6bb6bbcc admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:56.748257 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac274c6f-a801-4e21-bee2-63eb6bb6bbcc admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:56.760730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b1cc5-af34-4444-a16d-569e4139e96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:56.765336 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b1cc5-af34-4444-a16d-569e4139e96f admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:56.766003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b1cc5-af34-4444-a16d-569e4139e96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:56.766526 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b1cc5-af34-4444-a16d-569e4139e96f admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:56.768091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b1cc5-af34-4444-a16d-569e4139e96f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:56.771165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b1cc5-af34-4444-a16d-569e4139e96f admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:56.938530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 849/1679] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:21:56 2022] POST /identity/v3/projects => generated 385 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:21:56.943455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df392a1e-0a93-4fac-a4ed-d6d65c89f272 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:56.973732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df392a1e-0a93-4fac-a4ed-d6d65c89f272 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:57.145652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26397867-71c0-4bbd-ade5-be4bcef78991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:57.146450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26397867-71c0-4bbd-ade5-be4bcef78991 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:57.146830 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26397867-71c0-4bbd-ade5-be4bcef78991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:57.147302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26397867-71c0-4bbd-ade5-be4bcef78991 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:57.149743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26397867-71c0-4bbd-ade5-be4bcef78991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:57.158665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26397867-71c0-4bbd-ade5-be4bcef78991 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:57.284548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 831/1680] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:21:56 2022] POST /identity/v3/users => generated 299 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:57.288792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b1cc5-af34-4444-a16d-569e4139e96f admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:57.322687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b1cc5-af34-4444-a16d-569e4139e96f admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:57.349601 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ee1ab-ddc4-4086-9fda-ee3c8c8d0e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:57.351868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ee1ab-ddc4-4086-9fda-ee3c8c8d0e55 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:57.352980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ee1ab-ddc4-4086-9fda-ee3c8c8d0e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:57.353482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ee1ab-ddc4-4086-9fda-ee3c8c8d0e55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:57.355085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ee1ab-ddc4-4086-9fda-ee3c8c8d0e55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:57.357661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ee1ab-ddc4-4086-9fda-ee3c8c8d0e55 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:57.403287 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26397867-71c0-4bbd-ade5-be4bcef78991 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:57.418142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 850/1681] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:57.446356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26397867-71c0-4bbd-ade5-be4bcef78991 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:57.483680 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d697499-ca3d-4762-b605-1c3ff88f89c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:57.486648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d697499-ca3d-4762-b605-1c3ff88f89c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:57.487798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d697499-ca3d-4762-b605-1c3ff88f89c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:57.494782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d697499-ca3d-4762-b605-1c3ff88f89c2 admin admin] PATH_INFO: `/v3/projects/79127cf7d3214e8aabec9a73999aae6f/users/59fdf169da7d49c68cc4318afeb6785b/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:57.552698 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d697499-ca3d-4762-b605-1c3ff88f89c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79127cf7d3214e8aabec9a73999aae6f, user_id=59fdf169da7d49c68cc4318afeb6785b, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:57.579313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d697499-ca3d-4762-b605-1c3ff88f89c2 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:57.603231 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0444a-dcd5-48bf-9de8-609c5202ac6b None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:57.603551 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0444a-dcd5-48bf-9de8-609c5202ac6b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:57.603892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0444a-dcd5-48bf-9de8-609c5202ac6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:57.679891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-6df0444a-dcd5-48bf-9de8-609c5202ac6b None None] MFA Rules not processed for user `4beff9397579496c80fd6e93b7a81542`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:57.685098 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6df0444a-dcd5-48bf-9de8-609c5202ac6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:57.736686 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 832/1682] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:57 2022] PUT /identity/v3/projects/79127cf7d3214e8aabec9a73999aae6f/users/59fdf169da7d49c68cc4318afeb6785b/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:57.738547 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d697499-ca3d-4762-b605-1c3ff88f89c2 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:57.758925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-7d697499-ca3d-4762-b605-1c3ff88f89c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:57.881440 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 851/1683] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:21:57.897868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0444a-dcd5-48bf-9de8-609c5202ac6b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:57.939527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0444a-dcd5-48bf-9de8-609c5202ac6b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:58.003676 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d697499-ca3d-4762-b605-1c3ff88f89c2 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:58.035611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489daf0c-f8f9-477f-a331-fd1610923cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:58.036735 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489daf0c-f8f9-477f-a331-fd1610923cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:58.037319 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489daf0c-f8f9-477f-a331-fd1610923cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:58.037691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489daf0c-f8f9-477f-a331-fd1610923cc8 admin admin] PATH_INFO: `/v3/projects/1220d5ef84ff490bb6afa6eb30d67537` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:58.044754 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489daf0c-f8f9-477f-a331-fd1610923cc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1220d5ef84ff490bb6afa6eb30d67537)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:58.050837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489daf0c-f8f9-477f-a331-fd1610923cc8 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:58.137701 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:58.138230 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:58.179563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fde18-ba05-4077-9538-dcc502e19f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:58.180382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fde18-ba05-4077-9538-dcc502e19f45 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:58.180678 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fde18-ba05-4077-9538-dcc502e19f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:58.181004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fde18-ba05-4077-9538-dcc502e19f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:58.182143 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8fde18-ba05-4077-9538-dcc502e19f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:58.183730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8fde18-ba05-4077-9538-dcc502e19f45 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:58.213554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 852/1684] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:57 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:58.213554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fde18-ba05-4077-9538-dcc502e19f45 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:58.217887 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-8c8fde18-ba05-4077-9538-dcc502e19f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:58.277673 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-489daf0c-f8f9-477f-a331-fd1610923cc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1220d5ef84ff490bb6afa6eb30d67537', 'request_id': 'req-489daf0c-f8f9-477f-a331-fd1610923cc8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:58.298589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-489daf0c-f8f9-477f-a331-fd1610923cc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1220d5ef84ff490bb6afa6eb30d67537', 'request_id': 'req-489daf0c-f8f9-477f-a331-fd1610923cc8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:58.389947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-489daf0c-f8f9-477f-a331-fd1610923cc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1220d5ef84ff490bb6afa6eb30d67537', 'request_id': 'req-489daf0c-f8f9-477f-a331-fd1610923cc8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:58.399877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fde18-ba05-4077-9538-dcc502e19f45 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:58.406351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-489daf0c-f8f9-477f-a331-fd1610923cc8 admin admin] The token cache is being invalidate because project 1220d5ef84ff490bb6afa6eb30d67537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:21:58.406956 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-489daf0c-f8f9-477f-a331-fd1610923cc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:58.408663 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-489daf0c-f8f9-477f-a331-fd1610923cc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:21:58.411009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 833/1685] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:21:57 2022] DELETE /identity/v3/projects/1220d5ef84ff490bb6afa6eb30d67537 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:58.412791 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489daf0c-f8f9-477f-a331-fd1610923cc8 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:58.415395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-489daf0c-f8f9-477f-a331-fd1610923cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:58.595964 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14d953-df55-4167-8197-d2d5261ec71b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:58.596681 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14d953-df55-4167-8197-d2d5261ec71b service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:58.597033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14d953-df55-4167-8197-d2d5261ec71b service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:58.597255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14d953-df55-4167-8197-d2d5261ec71b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:58.605556 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-dc14d953-df55-4167-8197-d2d5261ec71b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:58.633099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489daf0c-f8f9-477f-a331-fd1610923cc8 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:58.653955 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbce45c-aed6-4095-a34c-c0a3a6b5e074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:58.655922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbce45c-aed6-4095-a34c-c0a3a6b5e074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:58.656472 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbce45c-aed6-4095-a34c-c0a3a6b5e074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:58.657013 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbce45c-aed6-4095-a34c-c0a3a6b5e074 admin admin] PATH_INFO: `/v3/projects/79127cf7d3214e8aabec9a73999aae6f/users/59fdf169da7d49c68cc4318afeb6785b/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:58.663772 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbce45c-aed6-4095-a34c-c0a3a6b5e074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79127cf7d3214e8aabec9a73999aae6f, user_id=59fdf169da7d49c68cc4318afeb6785b, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:58.670411 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbce45c-aed6-4095-a34c-c0a3a6b5e074 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:58.710499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 834/1686] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:58 2022] PUT /identity/v3/projects/79127cf7d3214e8aabec9a73999aae6f/users/59fdf169da7d49c68cc4318afeb6785b/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:21:58.714634 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a986c-1f4f-4ec5-82f1-70d487100c91 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:58.716382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a986c-1f4f-4ec5-82f1-70d487100c91 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:58.717841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a986c-1f4f-4ec5-82f1-70d487100c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:58.764697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-619a986c-1f4f-4ec5-82f1-70d487100c91 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:58.766230 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-619a986c-1f4f-4ec5-82f1-70d487100c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:58.790516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14d953-df55-4167-8197-d2d5261ec71b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:58.792540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14d953-df55-4167-8197-d2d5261ec71b service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:58.827057 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 853/1687] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:21:58.831127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372d213-3a9b-4d37-8bc1-cdb747f3627c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:58.831463 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372d213-3a9b-4d37-8bc1-cdb747f3627c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:58.831882 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372d213-3a9b-4d37-8bc1-cdb747f3627c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:58.857273 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:58.927182 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-b372d213-3a9b-4d37-8bc1-cdb747f3627c None None] MFA Rules not processed for user `59fdf169da7d49c68cc4318afeb6785b`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:58.948062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 835/1688] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:58.974798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88b30c-ab57-46f9-a2d4-9836134c903e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:58.976989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88b30c-ab57-46f9-a2d4-9836134c903e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:58.977330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88b30c-ab57-46f9-a2d4-9836134c903e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:58.981344 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b372d213-3a9b-4d37-8bc1-cdb747f3627c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:59.034720 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-9b88b30c-ab57-46f9-a2d4-9836134c903e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:21:59.035968 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9b88b30c-ab57-46f9-a2d4-9836134c903e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:59.071131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 836/1689] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:59.072210 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88b30c-ab57-46f9-a2d4-9836134c903e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:59.104422 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88b30c-ab57-46f9-a2d4-9836134c903e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:59.121999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba190260-010e-4cce-aa92-a9c2d300ea0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:59.122260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba190260-010e-4cce-aa92-a9c2d300ea0a admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:59.122473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba190260-010e-4cce-aa92-a9c2d300ea0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:59.122754 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba190260-010e-4cce-aa92-a9c2d300ea0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:59.123747 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba190260-010e-4cce-aa92-a9c2d300ea0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:21:59.124066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba190260-010e-4cce-aa92-a9c2d300ea0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:59.127438 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba190260-010e-4cce-aa92-a9c2d300ea0a admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:59.139239 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 854/1690] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:21:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:21:59.163459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372d213-3a9b-4d37-8bc1-cdb747f3627c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:59.165863 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b372d213-3a9b-4d37-8bc1-cdb747f3627c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:21:59.169893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 837/1691] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:21:59.178165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba190260-010e-4cce-aa92-a9c2d300ea0a admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:59.204805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba190260-010e-4cce-aa92-a9c2d300ea0a admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:59.224535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adad29f-60b3-4486-9724-dc7c36c563a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:59.224535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adad29f-60b3-4486-9724-dc7c36c563a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:59.224535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adad29f-60b3-4486-9724-dc7c36c563a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:59.224535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adad29f-60b3-4486-9724-dc7c36c563a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:59.226563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adad29f-60b3-4486-9724-dc7c36c563a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:59.231344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adad29f-60b3-4486-9724-dc7c36c563a4 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:59.315590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372d213-3a9b-4d37-8bc1-cdb747f3627c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:59.329120 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee033b47-2081-4afa-9a9a-3523aff5b606 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:59.329963 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee033b47-2081-4afa-9a9a-3523aff5b606 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:59.330359 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee033b47-2081-4afa-9a9a-3523aff5b606 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:59.332028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee033b47-2081-4afa-9a9a-3523aff5b606 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:59.352807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee033b47-2081-4afa-9a9a-3523aff5b606 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:59.357847 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:21:59.371982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 838/1692] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:21:59 2022] POST /identity/v3/projects => generated 409 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:59.373611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adad29f-60b3-4486-9724-dc7c36c563a4 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:59.375366 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee033b47-2081-4afa-9a9a-3523aff5b606 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:59.406479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 855/1693] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:21:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:21:59.534563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adad29f-60b3-4486-9724-dc7c36c563a4 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:59.562630 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267d355-326a-4243-887e-4e9a4bf2757e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:59.566295 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267d355-326a-4243-887e-4e9a4bf2757e admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:59.568652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267d355-326a-4243-887e-4e9a4bf2757e admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:59.569314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267d355-326a-4243-887e-4e9a4bf2757e admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:59.570875 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267d355-326a-4243-887e-4e9a4bf2757e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:59.576685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267d355-326a-4243-887e-4e9a4bf2757e admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:59.719625 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 839/1694] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:21:59 2022] POST /identity/v3/users => generated 305 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:21:59.724587 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee033b47-2081-4afa-9a9a-3523aff5b606 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:59.742550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee033b47-2081-4afa-9a9a-3523aff5b606 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:59.761612 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3623b10-2191-4dd7-8a3f-cf1329ae908c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:59.764289 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3623b10-2191-4dd7-8a3f-cf1329ae908c admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:59.764825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3623b10-2191-4dd7-8a3f-cf1329ae908c admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:59.765295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3623b10-2191-4dd7-8a3f-cf1329ae908c admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:59.766492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3623b10-2191-4dd7-8a3f-cf1329ae908c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:59.773059 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3623b10-2191-4dd7-8a3f-cf1329ae908c admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:59.811298 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 856/1695] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:59.816891 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267d355-326a-4243-887e-4e9a4bf2757e admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:59.835686 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267d355-326a-4243-887e-4e9a4bf2757e admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:21:59.900378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9622b-3b0e-47c5-93f3-19d90ca4d25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:21:59.901115 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9622b-3b0e-47c5-93f3-19d90ca4d25c admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:21:59.903505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9622b-3b0e-47c5-93f3-19d90ca4d25c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:21:59.903821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9622b-3b0e-47c5-93f3-19d90ca4d25c admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:21:59.904848 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a9622b-3b0e-47c5-93f3-19d90ca4d25c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:21:59.906722 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a9622b-3b0e-47c5-93f3-19d90ca4d25c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:21:59.949868 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 840/1696] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:21:59.950880 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3623b10-2191-4dd7-8a3f-cf1329ae908c admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:21:59.996124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3623b10-2191-4dd7-8a3f-cf1329ae908c admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:00.029736 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a94c553-e872-4258-a5b8-95b4403221ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:00.030809 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94c553-e872-4258-a5b8-95b4403221ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:00.031101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94c553-e872-4258-a5b8-95b4403221ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:00.031387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94c553-e872-4258-a5b8-95b4403221ef admin admin] PATH_INFO: `/v3/projects/92f3d6797c4f4925b39d9446e3efc38d/users/73a7ebadbe5e41e7a5aecac556c3fc90/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:00.078073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a94c553-e872-4258-a5b8-95b4403221ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f3d6797c4f4925b39d9446e3efc38d, user_id=73a7ebadbe5e41e7a5aecac556c3fc90, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:00.083127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a94c553-e872-4258-a5b8-95b4403221ef admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:00.206674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 857/1697] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:21:59 2022] PUT /identity/v3/projects/92f3d6797c4f4925b39d9446e3efc38d/users/73a7ebadbe5e41e7a5aecac556c3fc90/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:00.211557 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90c3b5-4d28-4eec-af9e-4f699947af6d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:00.211926 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90c3b5-4d28-4eec-af9e-4f699947af6d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:00.212215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90c3b5-4d28-4eec-af9e-4f699947af6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:00.313713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-7a90c3b5-4d28-4eec-af9e-4f699947af6d None None] MFA Rules not processed for user `73a7ebadbe5e41e7a5aecac556c3fc90`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:00.315638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-7a90c3b5-4d28-4eec-af9e-4f699947af6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:00.486265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ce574-4e89-4998-81cf-7589829e42c4 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:00.486563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ce574-4e89-4998-81cf-7589829e42c4 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:00.486877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ce574-4e89-4998-81cf-7589829e42c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:00.504533 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 841/1698] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:00 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:00.510653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90c3b5-4d28-4eec-af9e-4f699947af6d None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:00.523444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-b78ce574-4e89-4998-81cf-7589829e42c4 None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:00.524705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b78ce574-4e89-4998-81cf-7589829e42c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:00.666950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 858/1699] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jun 15 10:22:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:00.729173 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90c3b5-4d28-4eec-af9e-4f699947af6d None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:00.741006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145cc74c-47ce-4091-b6c3-77a273be48cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:00.744203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cc74c-47ce-4091-b6c3-77a273be48cc admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:00.744860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cc74c-47ce-4091-b6c3-77a273be48cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:00.745955 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cc74c-47ce-4091-b6c3-77a273be48cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:00.749147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145cc74c-47ce-4091-b6c3-77a273be48cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:00.754409 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145cc74c-47ce-4091-b6c3-77a273be48cc admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:00.798353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:00.949149 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ce574-4e89-4998-81cf-7589829e42c4 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:00.951643 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 842/1700] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:00 2022] POST /identity/v3/projects => generated 407 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:00.961015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:00.968715 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ce574-4e89-4998-81cf-7589829e42c4 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:01.142943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e577b9-c0f0-4748-bde4-2ee3aaff3ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:01.144410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e577b9-c0f0-4748-bde4-2ee3aaff3ba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:01.144846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e577b9-c0f0-4748-bde4-2ee3aaff3ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:01.147590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e577b9-c0f0-4748-bde4-2ee3aaff3ba1 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:01.150241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e577b9-c0f0-4748-bde4-2ee3aaff3ba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:01.156004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e577b9-c0f0-4748-bde4-2ee3aaff3ba1 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:01.281103 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145cc74c-47ce-4091-b6c3-77a273be48cc admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:01.284108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-145cc74c-47ce-4091-b6c3-77a273be48cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:01.297506 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 859/1701] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:00 2022] POST /identity/v3/users => generated 310 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:01.302337 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e577b9-c0f0-4748-bde4-2ee3aaff3ba1 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:01.337923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e577b9-c0f0-4748-bde4-2ee3aaff3ba1 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:01.372511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7ab34-dbfe-4d15-b00d-c74715d2bd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:01.373252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7ab34-dbfe-4d15-b00d-c74715d2bd5a admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:01.373543 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7ab34-dbfe-4d15-b00d-c74715d2bd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:01.373952 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7ab34-dbfe-4d15-b00d-c74715d2bd5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:01.374659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7ab34-dbfe-4d15-b00d-c74715d2bd5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:01.378285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7ab34-dbfe-4d15-b00d-c74715d2bd5a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:01.408189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 860/1702] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:01.410078 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7ab34-dbfe-4d15-b00d-c74715d2bd5a admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:01.443711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7ab34-dbfe-4d15-b00d-c74715d2bd5a admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:01.455173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1cc94-51b9-4ccb-950e-ce631dcdfef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:01.457707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1cc94-51b9-4ccb-950e-ce631dcdfef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:01.470259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1cc94-51b9-4ccb-950e-ce631dcdfef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:01.470555 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1cc94-51b9-4ccb-950e-ce631dcdfef7 admin admin] PATH_INFO: `/v3/projects/8d79d88f5c294edc894812bea371fc3a/users/b65980304cd04162ab09e1a87dbeb6be/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:01.505536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1cc94-51b9-4ccb-950e-ce631dcdfef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d79d88f5c294edc894812bea371fc3a, user_id=b65980304cd04162ab09e1a87dbeb6be, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:01.509133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1cc94-51b9-4ccb-950e-ce631dcdfef7 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:01.513428 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145cc74c-47ce-4091-b6c3-77a273be48cc admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:01.527643 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a0dde-6a8a-46c4-a135-63cc5f28dced service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:01.528987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a0dde-6a8a-46c4-a135-63cc5f28dced service swift] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:01.529597 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a0dde-6a8a-46c4-a135-63cc5f28dced service swift] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:01.530285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a0dde-6a8a-46c4-a135-63cc5f28dced service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:01.628405 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 861/1703] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:01 2022] PUT /identity/v3/projects/8d79d88f5c294edc894812bea371fc3a/users/b65980304cd04162ab09e1a87dbeb6be/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:01.629644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1cc94-51b9-4ccb-950e-ce631dcdfef7 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:01.809575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a0dde-6a8a-46c4-a135-63cc5f28dced service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:01.810835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a0dde-6a8a-46c4-a135-63cc5f28dced service swift] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:01.853248 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 843/1704] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 10:22:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:22:01.856712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a0dde-6a8a-46c4-a135-63cc5f28dced service swift] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:01.908439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a0dde-6a8a-46c4-a135-63cc5f28dced service swift] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:01.929492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1cc94-51b9-4ccb-950e-ce631dcdfef7 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:01.962076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca8cd3-d22b-420e-b6b9-d41dcb2df555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:01.963102 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca8cd3-d22b-420e-b6b9-d41dcb2df555 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:01.966018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca8cd3-d22b-420e-b6b9-d41dcb2df555 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:01.967047 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca8cd3-d22b-420e-b6b9-d41dcb2df555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:01.977200 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e0ca8cd3-d22b-420e-b6b9-d41dcb2df555 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:02.198073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca8cd3-d22b-420e-b6b9-d41dcb2df555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:02.199603 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca8cd3-d22b-420e-b6b9-d41dcb2df555 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:02.241544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 862/1705] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:02.250248 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca8cd3-d22b-420e-b6b9-d41dcb2df555 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:02.250413 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c8312-36c9-4dcd-bbd5-82976348a3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:02.250508 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e0ca8cd3-d22b-420e-b6b9-d41dcb2df555 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:02.250578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c8312-36c9-4dcd-bbd5-82976348a3cf admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:02.250578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c8312-36c9-4dcd-bbd5-82976348a3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:02.250578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c8312-36c9-4dcd-bbd5-82976348a3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:02.250578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c8312-36c9-4dcd-bbd5-82976348a3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:02.258647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c8312-36c9-4dcd-bbd5-82976348a3cf admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:02.296219 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca8cd3-d22b-420e-b6b9-d41dcb2df555 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:02.310111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 844/1706] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:02.314257 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63073584-e07c-4b13-bbe8-ae8a5e2cda8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:02.316580 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63073584-e07c-4b13-bbe8-ae8a5e2cda8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:02.318015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63073584-e07c-4b13-bbe8-ae8a5e2cda8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:02.325793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c8312-36c9-4dcd-bbd5-82976348a3cf admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:02.325930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63073584-e07c-4b13-bbe8-ae8a5e2cda8a admin admin] PATH_INFO: `/v3/users/4612504f393546dfbea12ffb3eed5efe` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:02.325930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63073584-e07c-4b13-bbe8-ae8a5e2cda8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4612504f393546dfbea12ffb3eed5efe)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:02.328685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63073584-e07c-4b13-bbe8-ae8a5e2cda8a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:02.345574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:02.347073 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c8312-36c9-4dcd-bbd5-82976348a3cf admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:02.372181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3987a8-7944-440d-8385-26dc4f774a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:02.374228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3987a8-7944-440d-8385-26dc4f774a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:02.378667 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3987a8-7944-440d-8385-26dc4f774a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:02.379325 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3987a8-7944-440d-8385-26dc4f774a61 admin admin] PATH_INFO: `/v3/projects/8d79d88f5c294edc894812bea371fc3a/users/b65980304cd04162ab09e1a87dbeb6be/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:02.394427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3987a8-7944-440d-8385-26dc4f774a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d79d88f5c294edc894812bea371fc3a, user_id=b65980304cd04162ab09e1a87dbeb6be, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:02.401979 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3987a8-7944-440d-8385-26dc4f774a61 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:02.441489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 845/1707] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:02 2022] PUT /identity/v3/projects/8d79d88f5c294edc894812bea371fc3a/users/b65980304cd04162ab09e1a87dbeb6be/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:02.456203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdeb6a-f789-472b-a335-17e7cab60531 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:02.462342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdeb6a-f789-472b-a335-17e7cab60531 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:02.462342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdeb6a-f789-472b-a335-17e7cab60531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:02.559204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-d8cdeb6a-f789-472b-a335-17e7cab60531 None None] MFA Rules not processed for user `b65980304cd04162ab09e1a87dbeb6be`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:02.561343 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d8cdeb6a-f789-472b-a335-17e7cab60531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:02.579979 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:02.598040 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-63073584-e07c-4b13-bbe8-ae8a5e2cda8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4612504f393546dfbea12ffb3eed5efe'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:02.631581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-63073584-e07c-4b13-bbe8-ae8a5e2cda8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4612504f393546dfbea12ffb3eed5efe'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:02.715798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-63073584-e07c-4b13-bbe8-ae8a5e2cda8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4612504f393546dfbea12ffb3eed5efe'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:02.732529 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-63073584-e07c-4b13-bbe8-ae8a5e2cda8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4612504f393546dfbea12ffb3eed5efe'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:02.733853 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-63073584-e07c-4b13-bbe8-ae8a5e2cda8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4612504f393546dfbea12ffb3eed5efe'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:02.762789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 863/1708] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:02 2022] DELETE /identity/v3/users/4612504f393546dfbea12ffb3eed5efe => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:02.767335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5499c7-fc3e-49eb-8b92-486d6ed90b1c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:02.769901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5499c7-fc3e-49eb-8b92-486d6ed90b1c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:02.772214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5499c7-fc3e-49eb-8b92-486d6ed90b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:02.853974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 846/1709] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:02 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:02.863442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-6a5499c7-fc3e-49eb-8b92-486d6ed90b1c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:02.865624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdeb6a-f789-472b-a335-17e7cab60531 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:02.870400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d8cdeb6a-f789-472b-a335-17e7cab60531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:02.873854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6a5499c7-fc3e-49eb-8b92-486d6ed90b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:03.170507 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdeb6a-f789-472b-a335-17e7cab60531 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:03.207911 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 864/1710] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:03.396211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff107fa-730e-4c25-8c79-f27cfba51945 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:03.401295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5499c7-fc3e-49eb-8b92-486d6ed90b1c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:03.403207 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6a5499c7-fc3e-49eb-8b92-486d6ed90b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:03.415777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff107fa-730e-4c25-8c79-f27cfba51945 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:03.416063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff107fa-730e-4c25-8c79-f27cfba51945 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:03.416387 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff107fa-730e-4c25-8c79-f27cfba51945 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:03.620662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5499c7-fc3e-49eb-8b92-486d6ed90b1c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:03.620662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d93bd9-3b9f-496d-93eb-d73f2d488e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:03.620662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d93bd9-3b9f-496d-93eb-d73f2d488e37 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:03.620662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d93bd9-3b9f-496d-93eb-d73f2d488e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:03.622173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d93bd9-3b9f-496d-93eb-d73f2d488e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:03.622173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d93bd9-3b9f-496d-93eb-d73f2d488e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:03.622173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d93bd9-3b9f-496d-93eb-d73f2d488e37 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:03.656494 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 865/1711] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:22:03.712589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff107fa-730e-4c25-8c79-f27cfba51945 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:03.712589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff107fa-730e-4c25-8c79-f27cfba51945 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:03.714378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e83b6-7c9d-496d-bf2a-964c4c8bb8b2 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:03.714378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e83b6-7c9d-496d-bf2a-964c4c8bb8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:03.714378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e83b6-7c9d-496d-bf2a-964c4c8bb8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:03.717804 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 847/1712] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:02 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 852 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:22:03.791226 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-879e83b6-7c9d-496d-bf2a-964c4c8bb8b2 None None] MFA Rules not processed for user `bbb69faf3ec54489a92fe87935f2f777`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:03.792484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-879e83b6-7c9d-496d-bf2a-964c4c8bb8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:03.878867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:04.054483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85807e5-cdde-4d18-ad62-211edaa91e3f None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:04.057253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85807e5-cdde-4d18-ad62-211edaa91e3f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:04.057766 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85807e5-cdde-4d18-ad62-211edaa91e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:04.140091 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:04.179855 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-c85807e5-cdde-4d18-ad62-211edaa91e3f None None] MFA Rules not processed for user `9f1e192f3eab42ee9f9ed5d2789e0d1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:04.198496 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c85807e5-cdde-4d18-ad62-211edaa91e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:04.218493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 866/1713] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 520 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:04.253859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e83b6-7c9d-496d-bf2a-964c4c8bb8b2 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:04.256088 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-879e83b6-7c9d-496d-bf2a-964c4c8bb8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:04.573814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 848/1714] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 521 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:04.581437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85807e5-cdde-4d18-ad62-211edaa91e3f None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:04.615267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85807e5-cdde-4d18-ad62-211edaa91e3f None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:04.635975 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c1c49-ed85-470a-ae51-73bcdc0c3631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:04.638690 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c1c49-ed85-470a-ae51-73bcdc0c3631 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:04.638690 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c1c49-ed85-470a-ae51-73bcdc0c3631 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:04.638690 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c1c49-ed85-470a-ae51-73bcdc0c3631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:04.639518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e83b6-7c9d-496d-bf2a-964c4c8bb8b2 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:04.642735 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-715c1c49-ed85-470a-ae51-73bcdc0c3631 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:04.654251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d3e39-8542-4f85-a4d4-1db13d2bbf50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:04.655216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d3e39-8542-4f85-a4d4-1db13d2bbf50 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:04.655545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d3e39-8542-4f85-a4d4-1db13d2bbf50 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:04.655829 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d3e39-8542-4f85-a4d4-1db13d2bbf50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:04.682228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d3e39-8542-4f85-a4d4-1db13d2bbf50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:04.684050 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d3e39-8542-4f85-a4d4-1db13d2bbf50 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:04.750322 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 867/1715] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:04.944712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c1c49-ed85-470a-ae51-73bcdc0c3631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:04.947095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c1c49-ed85-470a-ae51-73bcdc0c3631 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:04.968861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 849/1716] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:04 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:04.979413 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d3e39-8542-4f85-a4d4-1db13d2bbf50 service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:04.997335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d3e39-8542-4f85-a4d4-1db13d2bbf50 service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:05.024446 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11692f0b-ddd4-436c-8d59-807846c945b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:05.025395 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11692f0b-ddd4-436c-8d59-807846c945b4 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:05.026036 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11692f0b-ddd4-436c-8d59-807846c945b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:05.044024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11692f0b-ddd4-436c-8d59-807846c945b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:05.082114 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11692f0b-ddd4-436c-8d59-807846c945b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:05.083373 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11692f0b-ddd4-436c-8d59-807846c945b4 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:05.146551 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 868/1717] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:05.293975 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb072bf3-6d24-4e6f-950f-9e9ea7a35e42 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:05.293975 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb072bf3-6d24-4e6f-950f-9e9ea7a35e42 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:05.293975 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb072bf3-6d24-4e6f-950f-9e9ea7a35e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:05.344154 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-fb072bf3-6d24-4e6f-950f-9e9ea7a35e42 None None] MFA Rules not processed for user `ae926ce83d464c768baddd90de31aa40`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:05.348315 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-fb072bf3-6d24-4e6f-950f-9e9ea7a35e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:05.374302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 850/1718] 10.222.0.58 () {58 vars in 1015 bytes} [Wed Jun 15 10:22:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:05.469839 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11692f0b-ddd4-436c-8d59-807846c945b4 service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:05.512993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11692f0b-ddd4-436c-8d59-807846c945b4 service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:05.542033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354bb64-2517-4f4e-88bf-90fe64690c54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:05.548338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354bb64-2517-4f4e-88bf-90fe64690c54 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:05.554099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354bb64-2517-4f4e-88bf-90fe64690c54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:05.554099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354bb64-2517-4f4e-88bf-90fe64690c54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:05.576153 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354bb64-2517-4f4e-88bf-90fe64690c54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:05.578362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354bb64-2517-4f4e-88bf-90fe64690c54 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:05.598689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 869/1719] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:07.153379 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2832c7-2dc1-465d-8e28-c954d832237a None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:07.154462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2832c7-2dc1-465d-8e28-c954d832237a None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:07.154901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2832c7-2dc1-465d-8e28-c954d832237a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:07.217226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-ac2832c7-2dc1-465d-8e28-c954d832237a None None] MFA Rules not processed for user `5090fa81be4d41bbb62da2c4f273d537`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:07.240918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ac2832c7-2dc1-465d-8e28-c954d832237a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:07.497265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 851/1720] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:07.502712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50298a64-eea7-43ef-9f38-534ec52e5009 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:07.503243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50298a64-eea7-43ef-9f38-534ec52e5009 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:07.503243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50298a64-eea7-43ef-9f38-534ec52e5009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:07.549484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-50298a64-eea7-43ef-9f38-534ec52e5009 None None] MFA Rules not processed for user `5090fa81be4d41bbb62da2c4f273d537`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:07.550922 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-50298a64-eea7-43ef-9f38-534ec52e5009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:07.586639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 870/1721] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:07.596026 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2832c7-2dc1-465d-8e28-c954d832237a None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:07.597981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ac2832c7-2dc1-465d-8e28-c954d832237a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:07.688519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2832c7-2dc1-465d-8e28-c954d832237a None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:07.707679 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bb004-9835-4a01-9c59-6b1531edf12f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:07.708554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bb004-9835-4a01-9c59-6b1531edf12f service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:07.708812 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bb004-9835-4a01-9c59-6b1531edf12f service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:07.709427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bb004-9835-4a01-9c59-6b1531edf12f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:07.735572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074bb004-9835-4a01-9c59-6b1531edf12f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:07.748432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074bb004-9835-4a01-9c59-6b1531edf12f service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:07.821335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 852/1722] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:07.905775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50298a64-eea7-43ef-9f38-534ec52e5009 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:07.907624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-50298a64-eea7-43ef-9f38-534ec52e5009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:07.938850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50298a64-eea7-43ef-9f38-534ec52e5009 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:07.953413 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e16bf-ad9a-495d-89c3-951ae6b49e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:07.958214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e16bf-ad9a-495d-89c3-951ae6b49e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:07.959360 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e16bf-ad9a-495d-89c3-951ae6b49e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:07.960423 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e16bf-ad9a-495d-89c3-951ae6b49e4d admin admin] PATH_INFO: `/v3/projects/cb38b3cc097e4d9ba642740438c88dfc` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:07.966705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e16bf-ad9a-495d-89c3-951ae6b49e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb38b3cc097e4d9ba642740438c88dfc)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:07.969645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e16bf-ad9a-495d-89c3-951ae6b49e4d admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:08.201136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e43e16bf-ad9a-495d-89c3-951ae6b49e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb38b3cc097e4d9ba642740438c88dfc', 'request_id': 'req-e43e16bf-ad9a-495d-89c3-951ae6b49e4d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:08.226613 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e43e16bf-ad9a-495d-89c3-951ae6b49e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb38b3cc097e4d9ba642740438c88dfc', 'request_id': 'req-e43e16bf-ad9a-495d-89c3-951ae6b49e4d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:08.250596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e43e16bf-ad9a-495d-89c3-951ae6b49e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb38b3cc097e4d9ba642740438c88dfc', 'request_id': 'req-e43e16bf-ad9a-495d-89c3-951ae6b49e4d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:08.251893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:08.348468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e43e16bf-ad9a-495d-89c3-951ae6b49e4d admin admin] The token cache is being invalidate because project cb38b3cc097e4d9ba642740438c88dfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:08.348468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e43e16bf-ad9a-495d-89c3-951ae6b49e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:08.348468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e43e16bf-ad9a-495d-89c3-951ae6b49e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:08.351827 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 871/1723] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:07 2022] DELETE /identity/v3/projects/cb38b3cc097e4d9ba642740438c88dfc => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:08.356736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add267b-56b7-4358-9f16-45a81b1b8c00 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:08.357355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add267b-56b7-4358-9f16-45a81b1b8c00 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:08.357355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add267b-56b7-4358-9f16-45a81b1b8c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:08.380459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:08.397102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-3add267b-56b7-4358-9f16-45a81b1b8c00 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:08.398722 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3add267b-56b7-4358-9f16-45a81b1b8c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:08.702565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 853/1724] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:08.716419 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383ea6d-95e8-4ba5-bd47-d56cd609e1a1 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:08.717619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383ea6d-95e8-4ba5-bd47-d56cd609e1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:08.718135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383ea6d-95e8-4ba5-bd47-d56cd609e1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:08.762299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-2383ea6d-95e8-4ba5-bd47-d56cd609e1a1 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:08.763917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2383ea6d-95e8-4ba5-bd47-d56cd609e1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:08.790785 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 872/1725] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:08.795180 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add267b-56b7-4358-9f16-45a81b1b8c00 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:08.824399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add267b-56b7-4358-9f16-45a81b1b8c00 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:08.832487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb5ba7-416b-4d9d-aa64-728076e3763c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:08.836855 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb5ba7-416b-4d9d-aa64-728076e3763c admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:08.843114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb5ba7-416b-4d9d-aa64-728076e3763c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:08.844287 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb5ba7-416b-4d9d-aa64-728076e3763c admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:08.846594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb5ba7-416b-4d9d-aa64-728076e3763c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:08.851076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb5ba7-416b-4d9d-aa64-728076e3763c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:08.857536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb5ba7-416b-4d9d-aa64-728076e3763c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:08.895014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 854/1726] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:08.899969 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383ea6d-95e8-4ba5-bd47-d56cd609e1a1 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:08.928873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383ea6d-95e8-4ba5-bd47-d56cd609e1a1 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:08.961438 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc509ff-eaf9-4ea5-8666-53cda1d94243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:08.962545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc509ff-eaf9-4ea5-8666-53cda1d94243 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:08.963178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc509ff-eaf9-4ea5-8666-53cda1d94243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:08.963610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc509ff-eaf9-4ea5-8666-53cda1d94243 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:08.964514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc509ff-eaf9-4ea5-8666-53cda1d94243 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:08.968757 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc509ff-eaf9-4ea5-8666-53cda1d94243 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:09.146314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 873/1727] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:08 2022] POST /identity/v3/projects => generated 367 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:09.150427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb5ba7-416b-4d9d-aa64-728076e3763c admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:09.167876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb5ba7-416b-4d9d-aa64-728076e3763c admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:09.479296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040d34c-7c74-40ae-8268-6aab9020f0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:09.479912 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040d34c-7c74-40ae-8268-6aab9020f0ed admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:09.480240 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040d34c-7c74-40ae-8268-6aab9020f0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:09.480545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040d34c-7c74-40ae-8268-6aab9020f0ed admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:09.481441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040d34c-7c74-40ae-8268-6aab9020f0ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:09.498148 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040d34c-7c74-40ae-8268-6aab9020f0ed admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:09.660542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 855/1728] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:09 2022] POST /identity/v3/users => generated 284 bytes in 511 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:22:09.677317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc509ff-eaf9-4ea5-8666-53cda1d94243 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:09.693951 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc509ff-eaf9-4ea5-8666-53cda1d94243 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:09.720426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c9c3d-bbec-4a23-851e-0958608522f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:09.721955 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c9c3d-bbec-4a23-851e-0958608522f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:09.723232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c9c3d-bbec-4a23-851e-0958608522f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:09.725435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c9c3d-bbec-4a23-851e-0958608522f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:09.726729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c9c3d-bbec-4a23-851e-0958608522f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:09.732240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c9c3d-bbec-4a23-851e-0958608522f0 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:09.760266 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 874/1729] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:09.765390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040d34c-7c74-40ae-8268-6aab9020f0ed admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:09.784410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040d34c-7c74-40ae-8268-6aab9020f0ed admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:09.814349 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8307a4-f523-4033-a949-4b06bf416acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:09.816082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8307a4-f523-4033-a949-4b06bf416acb admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:09.816550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8307a4-f523-4033-a949-4b06bf416acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:09.817260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8307a4-f523-4033-a949-4b06bf416acb admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:09.818199 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8307a4-f523-4033-a949-4b06bf416acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:09.820096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8307a4-f523-4033-a949-4b06bf416acb admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:09.853106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 856/1730] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:09.857863 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c9c3d-bbec-4a23-851e-0958608522f0 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:09.875319 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c9c3d-bbec-4a23-851e-0958608522f0 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:09.886359 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9daecb-ef86-4d54-ad80-56df2ef4d48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:09.887295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9daecb-ef86-4d54-ad80-56df2ef4d48d admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:09.889155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9daecb-ef86-4d54-ad80-56df2ef4d48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:09.889155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9daecb-ef86-4d54-ad80-56df2ef4d48d admin admin] PATH_INFO: `/v3/projects/76207cb13c7b4878954698bd438ee692/users/84801620ec7e470196e2255bc5b08424/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:09.922721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9daecb-ef86-4d54-ad80-56df2ef4d48d admin admin] RBAC: Authorizing `identity:create_grant(project_id=76207cb13c7b4878954698bd438ee692, user_id=84801620ec7e470196e2255bc5b08424, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:09.927704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9daecb-ef86-4d54-ad80-56df2ef4d48d admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:10.015991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 875/1731] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:09 2022] PUT /identity/v3/projects/76207cb13c7b4878954698bd438ee692/users/84801620ec7e470196e2255bc5b08424/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:10.028999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce18d2-037b-4bff-91d2-03604e405756 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:10.029716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce18d2-037b-4bff-91d2-03604e405756 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:10.030178 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce18d2-037b-4bff-91d2-03604e405756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:10.100845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-78ce18d2-037b-4bff-91d2-03604e405756 None None] MFA Rules not processed for user `84801620ec7e470196e2255bc5b08424`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:10.102576 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-78ce18d2-037b-4bff-91d2-03604e405756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:10.230702 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24a7da-be0e-4ec3-8ece-08e87d937fca None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:10.230991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24a7da-be0e-4ec3-8ece-08e87d937fca None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:10.231275 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24a7da-be0e-4ec3-8ece-08e87d937fca None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:10.235152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 876/1732] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jun 15 10:22:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:22:10.291727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 857/1733] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:10.304869 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4e25f-86b8-42e7-b290-34014690c78e None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:10.305248 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4e25f-86b8-42e7-b290-34014690c78e None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:10.305819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4e25f-86b8-42e7-b290-34014690c78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:10.317215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce18d2-037b-4bff-91d2-03604e405756 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:10.318775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-78ce18d2-037b-4bff-91d2-03604e405756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:10.325414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:10.355449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-04a4e25f-86b8-42e7-b290-34014690c78e None None] MFA Rules not processed for user `26f23f8595fc43ad94aee7d0192164a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:10.361025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-04a4e25f-86b8-42e7-b290-34014690c78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:10.515467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce18d2-037b-4bff-91d2-03604e405756 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:10.530149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0467e12-0fdb-456a-9821-ffa103b77cd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:10.531310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0467e12-0fdb-456a-9821-ffa103b77cd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:10.531798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0467e12-0fdb-456a-9821-ffa103b77cd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:10.532286 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0467e12-0fdb-456a-9821-ffa103b77cd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:10.573739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0467e12-0fdb-456a-9821-ffa103b77cd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:10.578050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0467e12-0fdb-456a-9821-ffa103b77cd5 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:10.618333 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 858/1734] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:10 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:22:10.620169 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564f96c-6683-437f-b4b6-021787a37038 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:10.622378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564f96c-6683-437f-b4b6-021787a37038 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:10.622851 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564f96c-6683-437f-b4b6-021787a37038 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:10.624592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 859/1735] 10.222.0.60 () {54 vars in 912 bytes} [Wed Jun 15 10:22:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:22:10.628524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 877/1736] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jun 15 10:22:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:10.637005 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55f32a-105b-4c31-804e-ff623f2fdd5a None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:10.637542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55f32a-105b-4c31-804e-ff623f2fdd5a None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:10.638370 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f55f32a-105b-4c31-804e-ff623f2fdd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:10.641402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4e25f-86b8-42e7-b290-34014690c78e None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:10.644484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-04a4e25f-86b8-42e7-b290-34014690c78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:10.692466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4e25f-86b8-42e7-b290-34014690c78e None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:10.693447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-6f55f32a-105b-4c31-804e-ff623f2fdd5a None None] MFA Rules not processed for user `ae926ce83d464c768baddd90de31aa40`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:10.695898 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-6f55f32a-105b-4c31-804e-ff623f2fdd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:10.708558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18d35c-02ea-4929-aee8-dada0c46b801 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:10.711610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18d35c-02ea-4929-aee8-dada0c46b801 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:10.712854 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18d35c-02ea-4929-aee8-dada0c46b801 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:10.716438 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18d35c-02ea-4929-aee8-dada0c46b801 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:10.757353 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f18d35c-02ea-4929-aee8-dada0c46b801 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:10.767230 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f18d35c-02ea-4929-aee8-dada0c46b801 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:10.846794 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 878/1737] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:10.850739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dc9e2-07e8-4d68-9f73-5a60f9ae7f80 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:10.851283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dc9e2-07e8-4d68-9f73-5a60f9ae7f80 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:10.859597 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dc9e2-07e8-4d68-9f73-5a60f9ae7f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:10.895104 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-713dc9e2-07e8-4d68-9f73-5a60f9ae7f80 None None] MFA Rules not processed for user `4beff9397579496c80fd6e93b7a81542`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:10.906707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-713dc9e2-07e8-4d68-9f73-5a60f9ae7f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:10.929563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 860/1738] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jun 15 10:22:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:10.944467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55f32a-105b-4c31-804e-ff623f2fdd5a None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:10.949444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-6f55f32a-105b-4c31-804e-ff623f2fdd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:10.986357 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f55f32a-105b-4c31-804e-ff623f2fdd5a None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:11.019367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd04e2-f6fc-4557-81e9-2d19a0c34deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:11.023498 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd04e2-f6fc-4557-81e9-2d19a0c34deb service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:11.029531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd04e2-f6fc-4557-81e9-2d19a0c34deb service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:11.030205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd04e2-f6fc-4557-81e9-2d19a0c34deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:11.063478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd04e2-f6fc-4557-81e9-2d19a0c34deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:11.070746 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd04e2-f6fc-4557-81e9-2d19a0c34deb service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:11.091695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 861/1739] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:22:11.122622 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 879/1740] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:12.282347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3a502-3b27-4813-ac1b-64b6811a6642 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:12.282796 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3a502-3b27-4813-ac1b-64b6811a6642 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:12.283000 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3a502-3b27-4813-ac1b-64b6811a6642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:12.332607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-ffd3a502-3b27-4813-ac1b-64b6811a6642 None None] MFA Rules not processed for user `5090fa81be4d41bbb62da2c4f273d537`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:12.333914 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ffd3a502-3b27-4813-ac1b-64b6811a6642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:12.448099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 862/1741] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:14.051558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb99785-a6e1-466d-ac18-161b28497d8b None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:14.052115 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb99785-a6e1-466d-ac18-161b28497d8b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:14.052187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb99785-a6e1-466d-ac18-161b28497d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:14.086008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-0bb99785-a6e1-466d-ac18-161b28497d8b None None] MFA Rules not processed for user `9f1e192f3eab42ee9f9ed5d2789e0d1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:14.089212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0bb99785-a6e1-466d-ac18-161b28497d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:14.112806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:14.225688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 880/1742] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:14.553434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddee1d3-c0a1-4e5b-9e05-80ee6177db8f None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:14.553831 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddee1d3-c0a1-4e5b-9e05-80ee6177db8f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:14.553952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddee1d3-c0a1-4e5b-9e05-80ee6177db8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:14.610535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-5ddee1d3-c0a1-4e5b-9e05-80ee6177db8f None None] MFA Rules not processed for user `bbb69faf3ec54489a92fe87935f2f777`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:14.614949 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-5ddee1d3-c0a1-4e5b-9e05-80ee6177db8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:14.868254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:14.935610 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 863/1743] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:16.017013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb99785-a6e1-466d-ac18-161b28497d8b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:16.020419 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0bb99785-a6e1-466d-ac18-161b28497d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:16.058176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb99785-a6e1-466d-ac18-161b28497d8b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:16.091079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef34365-60b0-405a-bf06-b34e3f80d65c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:16.096546 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef34365-60b0-405a-bf06-b34e3f80d65c service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:16.099943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef34365-60b0-405a-bf06-b34e3f80d65c service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:16.101553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef34365-60b0-405a-bf06-b34e3f80d65c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:16.111906 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-4ef34365-60b0-405a-bf06-b34e3f80d65c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:16.330651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef34365-60b0-405a-bf06-b34e3f80d65c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:16.332514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef34365-60b0-405a-bf06-b34e3f80d65c service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:16.358222 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 881/1744] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:21.801793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddee1d3-c0a1-4e5b-9e05-80ee6177db8f None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:21.803911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-5ddee1d3-c0a1-4e5b-9e05-80ee6177db8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:21.924427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddee1d3-c0a1-4e5b-9e05-80ee6177db8f None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:21.930267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f54b0-f6ab-419f-877d-43ad237cb2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:21.931215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f54b0-f6ab-419f-877d-43ad237cb2b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:21.931497 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f54b0-f6ab-419f-877d-43ad237cb2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:21.931772 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f54b0-f6ab-419f-877d-43ad237cb2b3 admin admin] PATH_INFO: `/v3/users/73a7ebadbe5e41e7a5aecac556c3fc90` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:21.933009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f54b0-f6ab-419f-877d-43ad237cb2b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a7ebadbe5e41e7a5aecac556c3fc90)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:21.935047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f54b0-f6ab-419f-877d-43ad237cb2b3 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:22.184550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-967f54b0-f6ab-419f-877d-43ad237cb2b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a7ebadbe5e41e7a5aecac556c3fc90'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:22.246047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-967f54b0-f6ab-419f-877d-43ad237cb2b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a7ebadbe5e41e7a5aecac556c3fc90'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:22.246639 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-967f54b0-f6ab-419f-877d-43ad237cb2b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a7ebadbe5e41e7a5aecac556c3fc90'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:22.247263 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-967f54b0-f6ab-419f-877d-43ad237cb2b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a7ebadbe5e41e7a5aecac556c3fc90'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:22.261418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-967f54b0-f6ab-419f-877d-43ad237cb2b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a7ebadbe5e41e7a5aecac556c3fc90'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:22.275029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 864/1745] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:21 2022] DELETE /identity/v3/users/73a7ebadbe5e41e7a5aecac556c3fc90 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:22.279699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5f010-5ff1-462c-bdce-e1f0318cd539 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:22.280411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5f010-5ff1-462c-bdce-e1f0318cd539 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:22.280411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5f010-5ff1-462c-bdce-e1f0318cd539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:22.297496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-0bc5f010-5ff1-462c-bdce-e1f0318cd539 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:22.298807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0bc5f010-5ff1-462c-bdce-e1f0318cd539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:22.385103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 882/1746] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:22.406186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f54b0-f6ab-419f-877d-43ad237cb2b3 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:22.416604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-967f54b0-f6ab-419f-877d-43ad237cb2b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:22.505760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f54b0-f6ab-419f-877d-43ad237cb2b3 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:22.517008 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b1caf-56a3-4f06-a595-48fd85f96261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:22.519001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b1caf-56a3-4f06-a595-48fd85f96261 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:22.519287 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b1caf-56a3-4f06-a595-48fd85f96261 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:22.519562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b1caf-56a3-4f06-a595-48fd85f96261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:22.534781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b1caf-56a3-4f06-a595-48fd85f96261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:22.536608 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b1caf-56a3-4f06-a595-48fd85f96261 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:22.550181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 865/1747] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:22.940463 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5f010-5ff1-462c-bdce-e1f0318cd539 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:22.943221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0bc5f010-5ff1-462c-bdce-e1f0318cd539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:22.967956 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5f010-5ff1-462c-bdce-e1f0318cd539 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:22.995161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd592cf-21fa-4025-8c3e-d9ecef235379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:22.997064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd592cf-21fa-4025-8c3e-d9ecef235379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:22.999435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd592cf-21fa-4025-8c3e-d9ecef235379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:22.999435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd592cf-21fa-4025-8c3e-d9ecef235379 admin admin] PATH_INFO: `/v3/users/88adc6706e4b4c339f14786c9ff24b68` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:23.001219 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd592cf-21fa-4025-8c3e-d9ecef235379 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88adc6706e4b4c339f14786c9ff24b68)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:23.006997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd592cf-21fa-4025-8c3e-d9ecef235379 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:23.279117 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-acd592cf-21fa-4025-8c3e-d9ecef235379 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88adc6706e4b4c339f14786c9ff24b68'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:23.289583 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:23.307929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-acd592cf-21fa-4025-8c3e-d9ecef235379 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88adc6706e4b4c339f14786c9ff24b68'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:23.393349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-acd592cf-21fa-4025-8c3e-d9ecef235379 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88adc6706e4b4c339f14786c9ff24b68'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:23.413958 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-acd592cf-21fa-4025-8c3e-d9ecef235379 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88adc6706e4b4c339f14786c9ff24b68'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:23.415612 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-acd592cf-21fa-4025-8c3e-d9ecef235379 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88adc6706e4b4c339f14786c9ff24b68'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:23.428122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 883/1748] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:22 2022] DELETE /identity/v3/users/88adc6706e4b4c339f14786c9ff24b68 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:23.432363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31336c0b-6da9-4ff4-9e8b-f4e35a7c40e5 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:23.434395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31336c0b-6da9-4ff4-9e8b-f4e35a7c40e5 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:23.434395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31336c0b-6da9-4ff4-9e8b-f4e35a7c40e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:23.451917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-31336c0b-6da9-4ff4-9e8b-f4e35a7c40e5 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:23.454871 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-31336c0b-6da9-4ff4-9e8b-f4e35a7c40e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:23.463116 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:23.572046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 866/1749] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:23.593365 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd592cf-21fa-4025-8c3e-d9ecef235379 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:23.594908 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-acd592cf-21fa-4025-8c3e-d9ecef235379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:23.684123 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd592cf-21fa-4025-8c3e-d9ecef235379 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:23.688606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4b91a-11c3-442d-9c23-445ad6961ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:23.691142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4b91a-11c3-442d-9c23-445ad6961ffe service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:23.691768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4b91a-11c3-442d-9c23-445ad6961ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:23.692117 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4b91a-11c3-442d-9c23-445ad6961ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:23.704114 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4b91a-11c3-442d-9c23-445ad6961ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:23.706771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4b91a-11c3-442d-9c23-445ad6961ffe service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:23.721300 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 884/1750] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:23.773604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31336c0b-6da9-4ff4-9e8b-f4e35a7c40e5 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:23.775386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-31336c0b-6da9-4ff4-9e8b-f4e35a7c40e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:23.794219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31336c0b-6da9-4ff4-9e8b-f4e35a7c40e5 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:23.802031 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c5344-e356-48c2-a77a-e0c896cc80db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:23.803476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c5344-e356-48c2-a77a-e0c896cc80db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:23.803965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c5344-e356-48c2-a77a-e0c896cc80db admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:23.804532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c5344-e356-48c2-a77a-e0c896cc80db admin admin] PATH_INFO: `/v3/projects/92f3d6797c4f4925b39d9446e3efc38d` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:23.824355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c5344-e356-48c2-a77a-e0c896cc80db admin admin] RBAC: Authorizing `identity:delete_project(project_id=92f3d6797c4f4925b39d9446e3efc38d)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:23.827308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886c5344-e356-48c2-a77a-e0c896cc80db admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:24.053920 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-886c5344-e356-48c2-a77a-e0c896cc80db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92f3d6797c4f4925b39d9446e3efc38d', 'request_id': 'req-886c5344-e356-48c2-a77a-e0c896cc80db'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:24.067259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-886c5344-e356-48c2-a77a-e0c896cc80db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92f3d6797c4f4925b39d9446e3efc38d', 'request_id': 'req-886c5344-e356-48c2-a77a-e0c896cc80db'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:24.163387 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-886c5344-e356-48c2-a77a-e0c896cc80db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92f3d6797c4f4925b39d9446e3efc38d', 'request_id': 'req-886c5344-e356-48c2-a77a-e0c896cc80db'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:24.188067 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-886c5344-e356-48c2-a77a-e0c896cc80db admin admin] The token cache is being invalidate because project 92f3d6797c4f4925b39d9446e3efc38d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:24.188562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-886c5344-e356-48c2-a77a-e0c896cc80db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:24.190636 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-886c5344-e356-48c2-a77a-e0c896cc80db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:24.192393 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 867/1751] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:23 2022] DELETE /identity/v3/projects/92f3d6797c4f4925b39d9446e3efc38d => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:24.197081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4b91a-11c3-442d-9c23-445ad6961ffe service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:24.214615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4b91a-11c3-442d-9c23-445ad6961ffe service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:24.333743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12146bbc-7319-41df-a15b-e795e63a0109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:24.336627 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12146bbc-7319-41df-a15b-e795e63a0109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:24.336928 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12146bbc-7319-41df-a15b-e795e63a0109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:24.337267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12146bbc-7319-41df-a15b-e795e63a0109 admin admin] PATH_INFO: `/v3/users/b65980304cd04162ab09e1a87dbeb6be` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:24.338742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12146bbc-7319-41df-a15b-e795e63a0109 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b65980304cd04162ab09e1a87dbeb6be)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:24.341490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12146bbc-7319-41df-a15b-e795e63a0109 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:24.562265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c5344-e356-48c2-a77a-e0c896cc80db admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:24.565756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-886c5344-e356-48c2-a77a-e0c896cc80db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:24.609505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-12146bbc-7319-41df-a15b-e795e63a0109 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b65980304cd04162ab09e1a87dbeb6be'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:24.615363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c5344-e356-48c2-a77a-e0c896cc80db admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:24.632617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-12146bbc-7319-41df-a15b-e795e63a0109 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b65980304cd04162ab09e1a87dbeb6be'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:24.632740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b869763-cd87-485b-8d88-b507bb15dc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:24.632740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b869763-cd87-485b-8d88-b507bb15dc2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:24.632740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b869763-cd87-485b-8d88-b507bb15dc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:24.632740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b869763-cd87-485b-8d88-b507bb15dc2b admin admin] PATH_INFO: `/v3/projects/7e4f842389d840738362f2594c3437f7` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:24.641876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b869763-cd87-485b-8d88-b507bb15dc2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e4f842389d840738362f2594c3437f7)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:24.645135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b869763-cd87-485b-8d88-b507bb15dc2b admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:24.708860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-12146bbc-7319-41df-a15b-e795e63a0109 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b65980304cd04162ab09e1a87dbeb6be'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:24.713407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:24.724739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-12146bbc-7319-41df-a15b-e795e63a0109 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b65980304cd04162ab09e1a87dbeb6be'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:24.725616 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-12146bbc-7319-41df-a15b-e795e63a0109 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b65980304cd04162ab09e1a87dbeb6be'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:24.735327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 885/1752] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:24 2022] DELETE /identity/v3/users/b65980304cd04162ab09e1a87dbeb6be => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:24.796054 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e825b-5885-4052-b161-88ae4fcb6390 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:24.796458 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e825b-5885-4052-b161-88ae4fcb6390 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:24.796918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e825b-5885-4052-b161-88ae4fcb6390 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:24.798999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 886/1753] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jun 15 10:22:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:22:24.807779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afd534-a198-43e2-ad40-899c9c88e5d6 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:24.811631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afd534-a198-43e2-ad40-899c9c88e5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:24.812133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afd534-a198-43e2-ad40-899c9c88e5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:24.909154 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8b869763-cd87-485b-8d88-b507bb15dc2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e4f842389d840738362f2594c3437f7', 'request_id': 'req-8b869763-cd87-485b-8d88-b507bb15dc2b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:24.911702 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-73afd534-a198-43e2-ad40-899c9c88e5d6 None None] MFA Rules not processed for user `26f23f8595fc43ad94aee7d0192164a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:24.914410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-73afd534-a198-43e2-ad40-899c9c88e5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:24.926200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8b869763-cd87-485b-8d88-b507bb15dc2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e4f842389d840738362f2594c3437f7', 'request_id': 'req-8b869763-cd87-485b-8d88-b507bb15dc2b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:25.008945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8b869763-cd87-485b-8d88-b507bb15dc2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e4f842389d840738362f2594c3437f7', 'request_id': 'req-8b869763-cd87-485b-8d88-b507bb15dc2b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:25.031218 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8b869763-cd87-485b-8d88-b507bb15dc2b admin admin] The token cache is being invalidate because project 7e4f842389d840738362f2594c3437f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:25.031525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8b869763-cd87-485b-8d88-b507bb15dc2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:25.034095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8b869763-cd87-485b-8d88-b507bb15dc2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:25.036802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 868/1754] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:24 2022] DELETE /identity/v3/projects/7e4f842389d840738362f2594c3437f7 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:25.040834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b869763-cd87-485b-8d88-b507bb15dc2b admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:25.042932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-8b869763-cd87-485b-8d88-b507bb15dc2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:25.103042 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:25.123682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 887/1755] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jun 15 10:22:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:25.160370 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b869763-cd87-485b-8d88-b507bb15dc2b admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:25.169022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e8a6a-612a-4577-b6e2-9fb77bf22b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:25.169982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e8a6a-612a-4577-b6e2-9fb77bf22b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:25.170273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e8a6a-612a-4577-b6e2-9fb77bf22b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:25.170554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e8a6a-612a-4577-b6e2-9fb77bf22b78 admin admin] PATH_INFO: `/v3/users/4c24d333f23d4718b9b954bf470b9871` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:25.172047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e8a6a-612a-4577-b6e2-9fb77bf22b78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c24d333f23d4718b9b954bf470b9871)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:25.175060 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e8a6a-612a-4577-b6e2-9fb77bf22b78 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:25.353905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afd534-a198-43e2-ad40-899c9c88e5d6 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:25.355561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-73afd534-a198-43e2-ad40-899c9c88e5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:25.370600 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afd534-a198-43e2-ad40-899c9c88e5d6 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:25.383719 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942a3a6-26e8-4dea-9ad1-b6921f9f6aa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:25.384743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942a3a6-26e8-4dea-9ad1-b6921f9f6aa6 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:25.385246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942a3a6-26e8-4dea-9ad1-b6921f9f6aa6 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:25.385797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942a3a6-26e8-4dea-9ad1-b6921f9f6aa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:25.387462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c44e8a6a-612a-4577-b6e2-9fb77bf22b78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c24d333f23d4718b9b954bf470b9871'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:25.403014 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942a3a6-26e8-4dea-9ad1-b6921f9f6aa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:25.404927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942a3a6-26e8-4dea-9ad1-b6921f9f6aa6 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:25.418351 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 888/1756] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:22:25.485849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c44e8a6a-612a-4577-b6e2-9fb77bf22b78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c24d333f23d4718b9b954bf470b9871'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:25.487270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c44e8a6a-612a-4577-b6e2-9fb77bf22b78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c24d333f23d4718b9b954bf470b9871'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:25.487954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c44e8a6a-612a-4577-b6e2-9fb77bf22b78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c24d333f23d4718b9b954bf470b9871'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:25.506459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c44e8a6a-612a-4577-b6e2-9fb77bf22b78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c24d333f23d4718b9b954bf470b9871'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:25.526798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 869/1757] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:25 2022] DELETE /identity/v3/users/4c24d333f23d4718b9b954bf470b9871 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:25.637886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942a3a6-26e8-4dea-9ad1-b6921f9f6aa6 service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:25.639424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-1942a3a6-26e8-4dea-9ad1-b6921f9f6aa6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:25.658784 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942a3a6-26e8-4dea-9ad1-b6921f9f6aa6 service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:25.792174 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9f0f0-d107-4a02-8ecd-6330a1b26c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:25.794524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9f0f0-d107-4a02-8ecd-6330a1b26c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:25.794885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9f0f0-d107-4a02-8ecd-6330a1b26c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:25.795235 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9f0f0-d107-4a02-8ecd-6330a1b26c63 admin admin] PATH_INFO: `/v3/projects/8d79d88f5c294edc894812bea371fc3a` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:25.798921 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9f0f0-d107-4a02-8ecd-6330a1b26c63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d79d88f5c294edc894812bea371fc3a)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:25.815561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9f0f0-d107-4a02-8ecd-6330a1b26c63 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:26.072496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-89d9f0f0-d107-4a02-8ecd-6330a1b26c63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d79d88f5c294edc894812bea371fc3a', 'request_id': 'req-89d9f0f0-d107-4a02-8ecd-6330a1b26c63'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:26.083878 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-89d9f0f0-d107-4a02-8ecd-6330a1b26c63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d79d88f5c294edc894812bea371fc3a', 'request_id': 'req-89d9f0f0-d107-4a02-8ecd-6330a1b26c63'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:26.100501 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-89d9f0f0-d107-4a02-8ecd-6330a1b26c63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d79d88f5c294edc894812bea371fc3a', 'request_id': 'req-89d9f0f0-d107-4a02-8ecd-6330a1b26c63'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:26.260345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:26.265561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-89d9f0f0-d107-4a02-8ecd-6330a1b26c63 admin admin] The token cache is being invalidate because project 8d79d88f5c294edc894812bea371fc3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:26.270027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-89d9f0f0-d107-4a02-8ecd-6330a1b26c63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:26.270027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-89d9f0f0-d107-4a02-8ecd-6330a1b26c63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:26.277061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 889/1758] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:25 2022] DELETE /identity/v3/projects/8d79d88f5c294edc894812bea371fc3a => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:26.278771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43a043-4fe5-45ea-a033-f34252f91602 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:26.279256 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43a043-4fe5-45ea-a033-f34252f91602 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:26.279361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43a043-4fe5-45ea-a033-f34252f91602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:26.304570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-0a43a043-4fe5-45ea-a033-f34252f91602 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:26.305965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-0a43a043-4fe5-45ea-a033-f34252f91602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:26.421668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 870/1759] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:26.514538 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd4e1f-6683-4a3b-a2d3-5d1e3d007424 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:26.515028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd4e1f-6683-4a3b-a2d3-5d1e3d007424 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:26.515105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd4e1f-6683-4a3b-a2d3-5d1e3d007424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:26.559005 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-5edd4e1f-6683-4a3b-a2d3-5d1e3d007424 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:26.560242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5edd4e1f-6683-4a3b-a2d3-5d1e3d007424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:26.570590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 890/1760] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:26.574486 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43a043-4fe5-45ea-a033-f34252f91602 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:26.589112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43a043-4fe5-45ea-a033-f34252f91602 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:26.594219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9233f-22a0-47ef-8b69-88a6c3fafaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:26.595224 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9233f-22a0-47ef-8b69-88a6c3fafaaa admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:26.596516 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9233f-22a0-47ef-8b69-88a6c3fafaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:26.597102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9233f-22a0-47ef-8b69-88a6c3fafaaa admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:26.598653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9233f-22a0-47ef-8b69-88a6c3fafaaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:26.599154 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9233f-22a0-47ef-8b69-88a6c3fafaaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:26.601570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9233f-22a0-47ef-8b69-88a6c3fafaaa admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:26.615157 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 871/1761] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:26.619632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd4e1f-6683-4a3b-a2d3-5d1e3d007424 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:26.630224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd4e1f-6683-4a3b-a2d3-5d1e3d007424 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:26.636908 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65ac4d-f7c7-4c6c-a539-6975bee68f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:26.637677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65ac4d-f7c7-4c6c-a539-6975bee68f79 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:26.637898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65ac4d-f7c7-4c6c-a539-6975bee68f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:26.638185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65ac4d-f7c7-4c6c-a539-6975bee68f79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:26.639561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65ac4d-f7c7-4c6c-a539-6975bee68f79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:26.641000 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b65ac4d-f7c7-4c6c-a539-6975bee68f79 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:27.444162 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 891/1762] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:26 2022] POST /identity/v3/projects => generated 381 bytes in 822 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:27.446423 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9233f-22a0-47ef-8b69-88a6c3fafaaa admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:27.458162 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9233f-22a0-47ef-8b69-88a6c3fafaaa admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:27.502011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:27.594300 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84cf14-205b-409c-b033-502372a67b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:27.594993 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84cf14-205b-409c-b033-502372a67b28 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:27.595375 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84cf14-205b-409c-b033-502372a67b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:27.596335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84cf14-205b-409c-b033-502372a67b28 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:27.597086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84cf14-205b-409c-b033-502372a67b28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:27.602464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84cf14-205b-409c-b033-502372a67b28 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:27.607417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65ac4d-f7c7-4c6c-a539-6975bee68f79 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:27.610201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2b65ac4d-f7c7-4c6c-a539-6975bee68f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:27.638682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65ac4d-f7c7-4c6c-a539-6975bee68f79 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:27.662599 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656b03a-a137-47ad-bf86-b261f369f226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:27.664067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656b03a-a137-47ad-bf86-b261f369f226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:27.664412 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656b03a-a137-47ad-bf86-b261f369f226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:27.664697 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656b03a-a137-47ad-bf86-b261f369f226 admin admin] PATH_INFO: `/v3/projects/3640e95659274ba3ad199bf69330750e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:27.666739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656b03a-a137-47ad-bf86-b261f369f226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3640e95659274ba3ad199bf69330750e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:27.669272 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656b03a-a137-47ad-bf86-b261f369f226 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:27.724239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 872/1763] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:27 2022] POST /identity/v3/users => generated 291 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:27.729528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84cf14-205b-409c-b033-502372a67b28 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:27.744833 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84cf14-205b-409c-b033-502372a67b28 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:27.752497 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b2ab1-8868-4c82-b852-b3bce389d4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:27.753788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b2ab1-8868-4c82-b852-b3bce389d4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:27.754145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b2ab1-8868-4c82-b852-b3bce389d4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:27.754539 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b2ab1-8868-4c82-b852-b3bce389d4a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:27.756591 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b2ab1-8868-4c82-b852-b3bce389d4a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:27.774020 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b2ab1-8868-4c82-b852-b3bce389d4a3 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:27.797704 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 873/1764] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:22:27.802629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b2ab1-8868-4c82-b852-b3bce389d4a3 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:27.837147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b2ab1-8868-4c82-b852-b3bce389d4a3 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:27.849787 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d656b03a-a137-47ad-bf86-b261f369f226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3640e95659274ba3ad199bf69330750e', 'request_id': 'req-d656b03a-a137-47ad-bf86-b261f369f226'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:27.870304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d656b03a-a137-47ad-bf86-b261f369f226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3640e95659274ba3ad199bf69330750e', 'request_id': 'req-d656b03a-a137-47ad-bf86-b261f369f226'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:27.884690 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d656b03a-a137-47ad-bf86-b261f369f226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3640e95659274ba3ad199bf69330750e', 'request_id': 'req-d656b03a-a137-47ad-bf86-b261f369f226'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:27.942382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5192b-22d9-493c-ae2c-1933d7afac1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:27.943582 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5192b-22d9-493c-ae2c-1933d7afac1d admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:27.944004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5192b-22d9-493c-ae2c-1933d7afac1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:27.944390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5192b-22d9-493c-ae2c-1933d7afac1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:27.946238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c5192b-22d9-493c-ae2c-1933d7afac1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:27.950472 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c5192b-22d9-493c-ae2c-1933d7afac1d admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:27.970447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 874/1765] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:27.972167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5192b-22d9-493c-ae2c-1933d7afac1d admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:27.987580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5192b-22d9-493c-ae2c-1933d7afac1d admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:27.993341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b959be-6883-440b-a5e2-e6b64e7a0c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:27.995150 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b959be-6883-440b-a5e2-e6b64e7a0c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:27.996475 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b959be-6883-440b-a5e2-e6b64e7a0c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:27.996779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b959be-6883-440b-a5e2-e6b64e7a0c99 admin admin] PATH_INFO: `/v3/projects/336f1ab55f494deab38bd7e505e67fda/users/13d3337fe4d847df9fcf95996582bec4/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:28.009085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d656b03a-a137-47ad-bf86-b261f369f226 admin admin] The token cache is being invalidate because project 3640e95659274ba3ad199bf69330750e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:28.009746 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d656b03a-a137-47ad-bf86-b261f369f226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:28.011554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d656b03a-a137-47ad-bf86-b261f369f226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:28.016671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 892/1766] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:27 2022] DELETE /identity/v3/projects/3640e95659274ba3ad199bf69330750e => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:28.024598 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656b03a-a137-47ad-bf86-b261f369f226 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:28.030711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d656b03a-a137-47ad-bf86-b261f369f226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:28.080908 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656b03a-a137-47ad-bf86-b261f369f226 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:28.086524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4e1a6-86af-4cf0-86c1-a24beaca974e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:28.087891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4e1a6-86af-4cf0-86c1-a24beaca974e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:28.088356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4e1a6-86af-4cf0-86c1-a24beaca974e admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:28.092982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4e1a6-86af-4cf0-86c1-a24beaca974e admin admin] PATH_INFO: `/v3/users/59fdf169da7d49c68cc4318afeb6785b` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:28.092982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4e1a6-86af-4cf0-86c1-a24beaca974e admin admin] RBAC: Authorizing `identity:delete_user(user_id=59fdf169da7d49c68cc4318afeb6785b)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:28.093520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4e1a6-86af-4cf0-86c1-a24beaca974e admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:28.248853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b959be-6883-440b-a5e2-e6b64e7a0c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=336f1ab55f494deab38bd7e505e67fda, user_id=13d3337fe4d847df9fcf95996582bec4, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:28.252450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b959be-6883-440b-a5e2-e6b64e7a0c99 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:28.349071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 875/1767] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:27 2022] PUT /identity/v3/projects/336f1ab55f494deab38bd7e505e67fda/users/13d3337fe4d847df9fcf95996582bec4/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:28.351027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d8939-bedd-4d11-bf57-f72ea1621d55 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:28.351225 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d8939-bedd-4d11-bf57-f72ea1621d55 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:28.351508 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d8939-bedd-4d11-bf57-f72ea1621d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:28.407019 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-9bc4e1a6-86af-4cf0-86c1-a24beaca974e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59fdf169da7d49c68cc4318afeb6785b'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:28.413542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-ce5d8939-bedd-4d11-bf57-f72ea1621d55 None None] MFA Rules not processed for user `13d3337fe4d847df9fcf95996582bec4`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:28.413542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ce5d8939-bedd-4d11-bf57-f72ea1621d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:28.427781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-9bc4e1a6-86af-4cf0-86c1-a24beaca974e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59fdf169da7d49c68cc4318afeb6785b'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:28.514876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 876/1768] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:28.519910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d8939-bedd-4d11-bf57-f72ea1621d55 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:28.523765 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ce5d8939-bedd-4d11-bf57-f72ea1621d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:28.551924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-9bc4e1a6-86af-4cf0-86c1-a24beaca974e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59fdf169da7d49c68cc4318afeb6785b'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:28.561643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-9bc4e1a6-86af-4cf0-86c1-a24beaca974e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59fdf169da7d49c68cc4318afeb6785b'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:28.563305 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-9bc4e1a6-86af-4cf0-86c1-a24beaca974e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59fdf169da7d49c68cc4318afeb6785b'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:28.580845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 893/1769] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:28 2022] DELETE /identity/v3/users/59fdf169da7d49c68cc4318afeb6785b => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:28.621629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d8939-bedd-4d11-bf57-f72ea1621d55 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:28.698822 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4f6b0-d594-4d1e-bd66-be3f8d786e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:28.699688 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4f6b0-d594-4d1e-bd66-be3f8d786e50 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:28.700218 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4f6b0-d594-4d1e-bd66-be3f8d786e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:28.700599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4f6b0-d594-4d1e-bd66-be3f8d786e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:28.701500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4f6b0-d594-4d1e-bd66-be3f8d786e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:28.704109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4f6b0-d594-4d1e-bd66-be3f8d786e50 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:28.805502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 877/1770] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:28 2022] POST /identity/v3/projects => generated 381 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:22:28.806617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4e1a6-86af-4cf0-86c1-a24beaca974e admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:28.808584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9bc4e1a6-86af-4cf0-86c1-a24beaca974e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:28.825913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4e1a6-86af-4cf0-86c1-a24beaca974e admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:28.906472 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba4f1f-442d-4e9a-a3c1-8f7a2fdfae5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:28.908590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba4f1f-442d-4e9a-a3c1-8f7a2fdfae5b admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:28.909003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba4f1f-442d-4e9a-a3c1-8f7a2fdfae5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:28.909572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba4f1f-442d-4e9a-a3c1-8f7a2fdfae5b admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:28.911366 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba4f1f-442d-4e9a-a3c1-8f7a2fdfae5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:28.915026 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba4f1f-442d-4e9a-a3c1-8f7a2fdfae5b admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:29.037956 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 894/1771] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:28 2022] POST /identity/v3/users => generated 297 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:29.041849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4f6b0-d594-4d1e-bd66-be3f8d786e50 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:29.056342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4f6b0-d594-4d1e-bd66-be3f8d786e50 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:29.069178 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b358e-6230-40e1-8abb-7887579f7cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:29.069987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b358e-6230-40e1-8abb-7887579f7cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:29.071266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b358e-6230-40e1-8abb-7887579f7cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:29.071266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b358e-6230-40e1-8abb-7887579f7cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:29.071266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b358e-6230-40e1-8abb-7887579f7cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:29.073039 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b358e-6230-40e1-8abb-7887579f7cfe admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:29.094694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 878/1772] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:29.099080 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba4f1f-442d-4e9a-a3c1-8f7a2fdfae5b admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:29.110612 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba4f1f-442d-4e9a-a3c1-8f7a2fdfae5b admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:29.116077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c7626-77d1-414f-9430-505a72ebe71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:29.117860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c7626-77d1-414f-9430-505a72ebe71e admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:29.125106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c7626-77d1-414f-9430-505a72ebe71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:29.125106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c7626-77d1-414f-9430-505a72ebe71e admin admin] PATH_INFO: `/v3/projects/0eeb804e7f1a44ee9032d443b1b53ef4/users/cfc7ab38096d4dcb8c5470db63ccc7b4/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:29.141072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c7626-77d1-414f-9430-505a72ebe71e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eeb804e7f1a44ee9032d443b1b53ef4, user_id=cfc7ab38096d4dcb8c5470db63ccc7b4, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:29.144602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c7626-77d1-414f-9430-505a72ebe71e admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:29.294861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 895/1773] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:29 2022] PUT /identity/v3/projects/0eeb804e7f1a44ee9032d443b1b53ef4/users/cfc7ab38096d4dcb8c5470db63ccc7b4/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:29.301478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b358e-6230-40e1-8abb-7887579f7cfe admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:29.312072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b358e-6230-40e1-8abb-7887579f7cfe admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:29.402258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe90365-4262-46cf-a35c-097ed9b68f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:29.403106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe90365-4262-46cf-a35c-097ed9b68f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:29.403396 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe90365-4262-46cf-a35c-097ed9b68f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:29.403739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe90365-4262-46cf-a35c-097ed9b68f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:29.405685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe90365-4262-46cf-a35c-097ed9b68f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:29.408716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe90365-4262-46cf-a35c-097ed9b68f99 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:29.428163 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 879/1774] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:29.429568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c7626-77d1-414f-9430-505a72ebe71e admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:29.439801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d50f21-3449-4c2f-91e5-fb13e32f35ed None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:29.440286 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d50f21-3449-4c2f-91e5-fb13e32f35ed None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:29.440890 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d50f21-3449-4c2f-91e5-fb13e32f35ed None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:29.443064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 880/1775] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jun 15 10:22:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:22:29.444288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c7626-77d1-414f-9430-505a72ebe71e admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:29.450412 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7efbb-b779-4af7-9531-6a9c9a35fe01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:29.451341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe79e6-fb04-4df9-96d4-26f1ce055b04 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:29.451735 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe79e6-fb04-4df9-96d4-26f1ce055b04 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:29.452084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe79e6-fb04-4df9-96d4-26f1ce055b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:29.452573 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7efbb-b779-4af7-9531-6a9c9a35fe01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:29.453115 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7efbb-b779-4af7-9531-6a9c9a35fe01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:29.453581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7efbb-b779-4af7-9531-6a9c9a35fe01 admin admin] PATH_INFO: `/v3/projects/0eeb804e7f1a44ee9032d443b1b53ef4/users/cfc7ab38096d4dcb8c5470db63ccc7b4/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:29.457004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc7efbb-b779-4af7-9531-6a9c9a35fe01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eeb804e7f1a44ee9032d443b1b53ef4, user_id=cfc7ab38096d4dcb8c5470db63ccc7b4, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:29.460785 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc7efbb-b779-4af7-9531-6a9c9a35fe01 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:29.485413 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-3ebe79e6-fb04-4df9-96d4-26f1ce055b04 None None] MFA Rules not processed for user `26f23f8595fc43ad94aee7d0192164a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:29.485413 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3ebe79e6-fb04-4df9-96d4-26f1ce055b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:29.486396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 896/1776] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:29 2022] PUT /identity/v3/projects/0eeb804e7f1a44ee9032d443b1b53ef4/users/cfc7ab38096d4dcb8c5470db63ccc7b4/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:29.488140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cd241-98b1-4be5-9be6-940e12a450a8 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:29.488585 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cd241-98b1-4be5-9be6-940e12a450a8 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:29.490071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41cd241-98b1-4be5-9be6-940e12a450a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:29.556851 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-e41cd241-98b1-4be5-9be6-940e12a450a8 None None] MFA Rules not processed for user `cfc7ab38096d4dcb8c5470db63ccc7b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:29.558918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e41cd241-98b1-4be5-9be6-940e12a450a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:29.612638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 881/1777] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jun 15 10:22:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:29.676741 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 897/1778] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:29.695412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe79e6-fb04-4df9-96d4-26f1ce055b04 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:29.697468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3ebe79e6-fb04-4df9-96d4-26f1ce055b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:29.840253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe79e6-fb04-4df9-96d4-26f1ce055b04 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:29.848138 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa728d-057f-444d-afe5-0b5fadc07f7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:29.848901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa728d-057f-444d-afe5-0b5fadc07f7a service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:29.849289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa728d-057f-444d-afe5-0b5fadc07f7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:29.849683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa728d-057f-444d-afe5-0b5fadc07f7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:29.862660 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa728d-057f-444d-afe5-0b5fadc07f7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:29.864376 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa728d-057f-444d-afe5-0b5fadc07f7a service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:29.893013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:29.909699 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 882/1779] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:29.929876 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cd241-98b1-4be5-9be6-940e12a450a8 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:29.933849 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e41cd241-98b1-4be5-9be6-940e12a450a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:29.958181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41cd241-98b1-4be5-9be6-940e12a450a8 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:29.967812 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c605b4-6970-47e4-99d2-5f78f05a9300 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:29.968694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c605b4-6970-47e4-99d2-5f78f05a9300 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:29.969028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c605b4-6970-47e4-99d2-5f78f05a9300 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:29.969323 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c605b4-6970-47e4-99d2-5f78f05a9300 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:29.980689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c605b4-6970-47e4-99d2-5f78f05a9300 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:29.982580 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c605b4-6970-47e4-99d2-5f78f05a9300 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:29.996345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 898/1780] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:30.008285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa728d-057f-444d-afe5-0b5fadc07f7a service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:30.010119 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a6fa728d-057f-444d-afe5-0b5fadc07f7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:30.180669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa728d-057f-444d-afe5-0b5fadc07f7a service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:30.185774 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c98de-1c94-4eb3-a403-4aebfb00c91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:30.187226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c98de-1c94-4eb3-a403-4aebfb00c91c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:30.187585 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c98de-1c94-4eb3-a403-4aebfb00c91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:30.187944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c98de-1c94-4eb3-a403-4aebfb00c91c admin admin] PATH_INFO: `/v3/projects/79127cf7d3214e8aabec9a73999aae6f` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:30.189344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c98de-1c94-4eb3-a403-4aebfb00c91c admin admin] RBAC: Authorizing `identity:delete_project(project_id=79127cf7d3214e8aabec9a73999aae6f)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:30.192906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c98de-1c94-4eb3-a403-4aebfb00c91c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:30.341956 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-279c98de-1c94-4eb3-a403-4aebfb00c91c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79127cf7d3214e8aabec9a73999aae6f', 'request_id': 'req-279c98de-1c94-4eb3-a403-4aebfb00c91c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:30.354866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-279c98de-1c94-4eb3-a403-4aebfb00c91c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79127cf7d3214e8aabec9a73999aae6f', 'request_id': 'req-279c98de-1c94-4eb3-a403-4aebfb00c91c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:30.443487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-279c98de-1c94-4eb3-a403-4aebfb00c91c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79127cf7d3214e8aabec9a73999aae6f', 'request_id': 'req-279c98de-1c94-4eb3-a403-4aebfb00c91c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:30.458902 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-279c98de-1c94-4eb3-a403-4aebfb00c91c admin admin] The token cache is being invalidate because project 79127cf7d3214e8aabec9a73999aae6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:30.459121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-279c98de-1c94-4eb3-a403-4aebfb00c91c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:30.459683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-279c98de-1c94-4eb3-a403-4aebfb00c91c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:30.461364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 883/1781] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:30 2022] DELETE /identity/v3/projects/79127cf7d3214e8aabec9a73999aae6f => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:30.467642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126e286-f6c4-455c-8e80-9e78b8fbc12c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:30.467995 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126e286-f6c4-455c-8e80-9e78b8fbc12c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:30.468280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126e286-f6c4-455c-8e80-9e78b8fbc12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:30.487771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-1126e286-f6c4-455c-8e80-9e78b8fbc12c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:30.489016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-1126e286-f6c4-455c-8e80-9e78b8fbc12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:30.557013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 899/1782] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:30.566964 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c43842-19b0-49d1-a9d7-ee33ff34e13e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:30.567362 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c43842-19b0-49d1-a9d7-ee33ff34e13e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:30.567753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c43842-19b0-49d1-a9d7-ee33ff34e13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:30.587858 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-87c43842-19b0-49d1-a9d7-ee33ff34e13e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:30.589179 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-87c43842-19b0-49d1-a9d7-ee33ff34e13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:30.597873 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 884/1783] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:30.613183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126e286-f6c4-455c-8e80-9e78b8fbc12c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:30.613315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126e286-f6c4-455c-8e80-9e78b8fbc12c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:30.618407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f7a20-3211-4df4-8acd-ab781c7cf9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:30.619166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f7a20-3211-4df4-8acd-ab781c7cf9e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:30.619450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f7a20-3211-4df4-8acd-ab781c7cf9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:30.619729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f7a20-3211-4df4-8acd-ab781c7cf9e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:30.620485 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f7a20-3211-4df4-8acd-ab781c7cf9e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:30.620798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f7a20-3211-4df4-8acd-ab781c7cf9e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:30.622475 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f7a20-3211-4df4-8acd-ab781c7cf9e1 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:30.633798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c43842-19b0-49d1-a9d7-ee33ff34e13e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:30.635241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-87c43842-19b0-49d1-a9d7-ee33ff34e13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:30.637304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 900/1784] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:30.644751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f7a20-3211-4df4-8acd-ab781c7cf9e1 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:30.650354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c43842-19b0-49d1-a9d7-ee33ff34e13e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:30.655770 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f7a20-3211-4df4-8acd-ab781c7cf9e1 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:30.657267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021a6c6-b3dc-4d25-800f-22be36568ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:30.658389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a6c6-b3dc-4d25-800f-22be36568ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:30.658676 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a6c6-b3dc-4d25-800f-22be36568ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:30.659464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021a6c6-b3dc-4d25-800f-22be36568ddd admin admin] PATH_INFO: `/v3/users/13d3337fe4d847df9fcf95996582bec4` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:30.661052 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021a6c6-b3dc-4d25-800f-22be36568ddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=13d3337fe4d847df9fcf95996582bec4)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:30.662283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd19ac-0422-44bf-a8ff-f7567aca0344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:30.662779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd19ac-0422-44bf-a8ff-f7567aca0344 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:30.663168 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd19ac-0422-44bf-a8ff-f7567aca0344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:30.663397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd19ac-0422-44bf-a8ff-f7567aca0344 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:30.663983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021a6c6-b3dc-4d25-800f-22be36568ddd admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:30.664096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd19ac-0422-44bf-a8ff-f7567aca0344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:30.665962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd19ac-0422-44bf-a8ff-f7567aca0344 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:30.779233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 901/1785] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:30 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:22:30.783562 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd19ac-0422-44bf-a8ff-f7567aca0344 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:30.854529 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd19ac-0422-44bf-a8ff-f7567aca0344 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:30.855623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-d021a6c6-b3dc-4d25-800f-22be36568ddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13d3337fe4d847df9fcf95996582bec4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:30.859875 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d7a52-ffe2-4acf-9344-a07ef6c503a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:30.860559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d7a52-ffe2-4acf-9344-a07ef6c503a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:30.861417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d7a52-ffe2-4acf-9344-a07ef6c503a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:30.861518 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:30.861700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257d7a52-ffe2-4acf-9344-a07ef6c503a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:30.862699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d7a52-ffe2-4acf-9344-a07ef6c503a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:30.864626 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257d7a52-ffe2-4acf-9344-a07ef6c503a2 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:30.986959 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-d021a6c6-b3dc-4d25-800f-22be36568ddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13d3337fe4d847df9fcf95996582bec4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:30.988209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-d021a6c6-b3dc-4d25-800f-22be36568ddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13d3337fe4d847df9fcf95996582bec4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:30.989313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-d021a6c6-b3dc-4d25-800f-22be36568ddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13d3337fe4d847df9fcf95996582bec4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:30.995633 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 902/1786] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:30 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:30.999340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d7a52-ffe2-4acf-9344-a07ef6c503a2 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:31.001070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-257d7a52-ffe2-4acf-9344-a07ef6c503a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:31.016218 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257d7a52-ffe2-4acf-9344-a07ef6c503a2 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:31.021522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-d021a6c6-b3dc-4d25-800f-22be36568ddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13d3337fe4d847df9fcf95996582bec4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:31.028187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a3d96-1857-42f0-93f5-f91800666254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:31.028890 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a3d96-1857-42f0-93f5-f91800666254 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:31.029233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a3d96-1857-42f0-93f5-f91800666254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:31.029514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a3d96-1857-42f0-93f5-f91800666254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:31.030465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a3d96-1857-42f0-93f5-f91800666254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:31.032463 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a3d96-1857-42f0-93f5-f91800666254 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:31.045227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 885/1787] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:30 2022] DELETE /identity/v3/users/13d3337fe4d847df9fcf95996582bec4 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:31.051535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68bd11-1cf5-4128-b52c-df37c324fb36 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:31.052063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68bd11-1cf5-4128-b52c-df37c324fb36 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:31.052445 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68bd11-1cf5-4128-b52c-df37c324fb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:31.058140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 903/1788] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:22:31.059175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a3d96-1857-42f0-93f5-f91800666254 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:31.072394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a3d96-1857-42f0-93f5-f91800666254 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:31.075077 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-cb68bd11-1cf5-4128-b52c-df37c324fb36 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:31.076476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-cb68bd11-1cf5-4128-b52c-df37c324fb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:31.166177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf97c1e-395d-4f75-8011-4f99902a2525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:31.166929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf97c1e-395d-4f75-8011-4f99902a2525 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:31.167252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf97c1e-395d-4f75-8011-4f99902a2525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:31.167453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf97c1e-395d-4f75-8011-4f99902a2525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:31.168275 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf97c1e-395d-4f75-8011-4f99902a2525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:31.169921 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf97c1e-395d-4f75-8011-4f99902a2525 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:31.170719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 886/1789] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:31.182260 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 904/1790] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:31.183653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68bd11-1cf5-4128-b52c-df37c324fb36 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:31.185939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-cb68bd11-1cf5-4128-b52c-df37c324fb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:31.186562 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf97c1e-395d-4f75-8011-4f99902a2525 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:31.199755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf97c1e-395d-4f75-8011-4f99902a2525 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:31.206830 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58850ee2-1bd4-4b7b-a19f-45c42154053b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:31.207830 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58850ee2-1bd4-4b7b-a19f-45c42154053b admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:31.208186 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58850ee2-1bd4-4b7b-a19f-45c42154053b admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:31.208553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58850ee2-1bd4-4b7b-a19f-45c42154053b admin admin] PATH_INFO: `/v3/projects/e4a070b8ebca4ed096879ff5077f8272/users/384b9998ce4945d3879362c03723cc3b/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:31.226213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58850ee2-1bd4-4b7b-a19f-45c42154053b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a070b8ebca4ed096879ff5077f8272, user_id=384b9998ce4945d3879362c03723cc3b, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:31.229005 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58850ee2-1bd4-4b7b-a19f-45c42154053b admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:31.252819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68bd11-1cf5-4128-b52c-df37c324fb36 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:31.256592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1b546-669b-4d58-a11b-24e7b5edc321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:31.257359 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1b546-669b-4d58-a11b-24e7b5edc321 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:31.257716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1b546-669b-4d58-a11b-24e7b5edc321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:31.258148 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1b546-669b-4d58-a11b-24e7b5edc321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:31.271038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1b546-669b-4d58-a11b-24e7b5edc321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:31.272541 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1b546-669b-4d58-a11b-24e7b5edc321 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:31.283611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 887/1791] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:31.320234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 905/1792] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:31 2022] PUT /identity/v3/projects/e4a070b8ebca4ed096879ff5077f8272/users/384b9998ce4945d3879362c03723cc3b/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:31.333553 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458d8b0-e19e-4397-833c-d7613ac17364 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:31.334056 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458d8b0-e19e-4397-833c-d7613ac17364 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:31.334529 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458d8b0-e19e-4397-833c-d7613ac17364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:31.378673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58850ee2-1bd4-4b7b-a19f-45c42154053b admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:31.380116 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-6458d8b0-e19e-4397-833c-d7613ac17364 None None] MFA Rules not processed for user `384b9998ce4945d3879362c03723cc3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:31.380396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-58850ee2-1bd4-4b7b-a19f-45c42154053b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:31.383637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-6458d8b0-e19e-4397-833c-d7613ac17364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:31.472266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 888/1793] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:31.478567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458d8b0-e19e-4397-833c-d7613ac17364 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:31.494896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58850ee2-1bd4-4b7b-a19f-45c42154053b admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:31.500439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b865717-b53d-4d89-bb9c-2d1a0ec2feb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:31.502243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b865717-b53d-4d89-bb9c-2d1a0ec2feb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:31.502718 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b865717-b53d-4d89-bb9c-2d1a0ec2feb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:31.503007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b865717-b53d-4d89-bb9c-2d1a0ec2feb6 admin admin] PATH_INFO: `/v3/users/84801620ec7e470196e2255bc5b08424` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:31.504334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b865717-b53d-4d89-bb9c-2d1a0ec2feb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84801620ec7e470196e2255bc5b08424)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:31.506335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b865717-b53d-4d89-bb9c-2d1a0ec2feb6 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:31.552416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458d8b0-e19e-4397-833c-d7613ac17364 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:31.557208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d45ce9-34cc-4317-8708-8ff0bc2f6780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:31.558498 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d45ce9-34cc-4317-8708-8ff0bc2f6780 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:31.558808 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d45ce9-34cc-4317-8708-8ff0bc2f6780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:31.559069 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d45ce9-34cc-4317-8708-8ff0bc2f6780 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:31.559764 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d45ce9-34cc-4317-8708-8ff0bc2f6780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:31.561707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d45ce9-34cc-4317-8708-8ff0bc2f6780 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:31.706465 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 889/1794] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:31 2022] POST /identity/v3/projects => generated 383 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:31.711679 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d45ce9-34cc-4317-8708-8ff0bc2f6780 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:31.812848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6b865717-b53d-4d89-bb9c-2d1a0ec2feb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84801620ec7e470196e2255bc5b08424'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:31.825549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d45ce9-34cc-4317-8708-8ff0bc2f6780 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:31.839295 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9af3e5-63eb-466a-9f84-7f1d7f47bece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:31.840830 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9af3e5-63eb-466a-9f84-7f1d7f47bece admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:31.842629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9af3e5-63eb-466a-9f84-7f1d7f47bece admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:31.843087 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9af3e5-63eb-466a-9f84-7f1d7f47bece admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:31.844186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9af3e5-63eb-466a-9f84-7f1d7f47bece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:31.847812 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6b865717-b53d-4d89-bb9c-2d1a0ec2feb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84801620ec7e470196e2255bc5b08424'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:31.849509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9af3e5-63eb-466a-9f84-7f1d7f47bece admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:31.994633 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6b865717-b53d-4d89-bb9c-2d1a0ec2feb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84801620ec7e470196e2255bc5b08424'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:32.008170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6b865717-b53d-4d89-bb9c-2d1a0ec2feb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84801620ec7e470196e2255bc5b08424'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:32.009397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6b865717-b53d-4d89-bb9c-2d1a0ec2feb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84801620ec7e470196e2255bc5b08424'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:32.022593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 906/1795] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:31 2022] DELETE /identity/v3/users/84801620ec7e470196e2255bc5b08424 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:32.025563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b865717-b53d-4d89-bb9c-2d1a0ec2feb6 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:32.026812 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6b865717-b53d-4d89-bb9c-2d1a0ec2feb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:32.089971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 890/1796] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:31 2022] POST /identity/v3/users => generated 298 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:22:32.091652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04760c-f2fe-4efb-8c28-2a616ea692f9 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:32.092123 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04760c-f2fe-4efb-8c28-2a616ea692f9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:32.092652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd04760c-f2fe-4efb-8c28-2a616ea692f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:32.115422 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-cd04760c-f2fe-4efb-8c28-2a616ea692f9 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:32.117139 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-cd04760c-f2fe-4efb-8c28-2a616ea692f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:32.126633 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b865717-b53d-4d89-bb9c-2d1a0ec2feb6 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:32.127574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 891/1797] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:32.128516 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04760c-f2fe-4efb-8c28-2a616ea692f9 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:32.134463 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241bfbc-772a-4354-989a-2cfb7a71b0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:32.135692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241bfbc-772a-4354-989a-2cfb7a71b0c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:32.135978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241bfbc-772a-4354-989a-2cfb7a71b0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:32.136259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241bfbc-772a-4354-989a-2cfb7a71b0c4 admin admin] PATH_INFO: `/v3/projects/336f1ab55f494deab38bd7e505e67fda` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:32.136352 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-cd04760c-f2fe-4efb-8c28-2a616ea692f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:32.137651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241bfbc-772a-4354-989a-2cfb7a71b0c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=336f1ab55f494deab38bd7e505e67fda)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:32.139772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241bfbc-772a-4354-989a-2cfb7a71b0c4 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:32.152843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd04760c-f2fe-4efb-8c28-2a616ea692f9 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:32.163175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78639c46-1b3f-41b4-9d01-dfecd5351794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:32.164079 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78639c46-1b3f-41b4-9d01-dfecd5351794 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:32.166055 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78639c46-1b3f-41b4-9d01-dfecd5351794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:32.166716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78639c46-1b3f-41b4-9d01-dfecd5351794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:32.168665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78639c46-1b3f-41b4-9d01-dfecd5351794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:32.170736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78639c46-1b3f-41b4-9d01-dfecd5351794 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:32.194259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 892/1798] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:32.194259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78639c46-1b3f-41b4-9d01-dfecd5351794 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:32.197334 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-78639c46-1b3f-41b4-9d01-dfecd5351794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:32.279122 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78639c46-1b3f-41b4-9d01-dfecd5351794 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:32.288450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df19ffb-c259-4e3c-9c3c-1747acb1fc98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:32.289499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df19ffb-c259-4e3c-9c3c-1747acb1fc98 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:32.289918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df19ffb-c259-4e3c-9c3c-1747acb1fc98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:32.290358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df19ffb-c259-4e3c-9c3c-1747acb1fc98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:32.302906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df19ffb-c259-4e3c-9c3c-1747acb1fc98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:32.304494 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df19ffb-c259-4e3c-9c3c-1747acb1fc98 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:32.316932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 893/1799] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:32.317822 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df19ffb-c259-4e3c-9c3c-1747acb1fc98 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:32.329127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df19ffb-c259-4e3c-9c3c-1747acb1fc98 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:32.351486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f241bfbc-772a-4354-989a-2cfb7a71b0c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '336f1ab55f494deab38bd7e505e67fda', 'request_id': 'req-f241bfbc-772a-4354-989a-2cfb7a71b0c4'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:32.362579 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f241bfbc-772a-4354-989a-2cfb7a71b0c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '336f1ab55f494deab38bd7e505e67fda', 'request_id': 'req-f241bfbc-772a-4354-989a-2cfb7a71b0c4'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:32.373838 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f241bfbc-772a-4354-989a-2cfb7a71b0c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '336f1ab55f494deab38bd7e505e67fda', 'request_id': 'req-f241bfbc-772a-4354-989a-2cfb7a71b0c4'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:32.448993 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fca3fb-349a-406e-bb19-32c2ae0c9497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:32.451355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fca3fb-349a-406e-bb19-32c2ae0c9497 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:32.451648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fca3fb-349a-406e-bb19-32c2ae0c9497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:32.451927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fca3fb-349a-406e-bb19-32c2ae0c9497 admin admin] PATH_INFO: `/v3/projects/997a2bb6b781476fb26522ef6c229e44/users/54fdcaab1da94b40acf08851e4d1841c/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:32.459399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f241bfbc-772a-4354-989a-2cfb7a71b0c4 admin admin] The token cache is being invalidate because project 336f1ab55f494deab38bd7e505e67fda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:32.459788 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f241bfbc-772a-4354-989a-2cfb7a71b0c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:32.463893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f241bfbc-772a-4354-989a-2cfb7a71b0c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:32.463893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 907/1800] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:32 2022] DELETE /identity/v3/projects/336f1ab55f494deab38bd7e505e67fda => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:32.468516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241bfbc-772a-4354-989a-2cfb7a71b0c4 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:32.474646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f241bfbc-772a-4354-989a-2cfb7a71b0c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:32.501183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fca3fb-349a-406e-bb19-32c2ae0c9497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=997a2bb6b781476fb26522ef6c229e44, user_id=54fdcaab1da94b40acf08851e4d1841c, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:32.501389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241bfbc-772a-4354-989a-2cfb7a71b0c4 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:32.504651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fca3fb-349a-406e-bb19-32c2ae0c9497 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:32.506687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6916c55-e998-4cd3-b38c-1a0d13887c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:32.507673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6916c55-e998-4cd3-b38c-1a0d13887c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:32.508076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6916c55-e998-4cd3-b38c-1a0d13887c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:32.509716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6916c55-e998-4cd3-b38c-1a0d13887c23 admin admin] PATH_INFO: `/v3/users/cfc7ab38096d4dcb8c5470db63ccc7b4` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:32.511912 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6916c55-e998-4cd3-b38c-1a0d13887c23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfc7ab38096d4dcb8c5470db63ccc7b4)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:32.514092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6916c55-e998-4cd3-b38c-1a0d13887c23 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:32.587123 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 894/1801] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:32 2022] PUT /identity/v3/projects/997a2bb6b781476fb26522ef6c229e44/users/54fdcaab1da94b40acf08851e4d1841c/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:32.591474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fca3fb-349a-406e-bb19-32c2ae0c9497 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:32.598215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-33fca3fb-349a-406e-bb19-32c2ae0c9497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:32.708393 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fca3fb-349a-406e-bb19-32c2ae0c9497 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:32.716427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99119f-0545-4904-b8af-ad5147ee2789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:32.718291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99119f-0545-4904-b8af-ad5147ee2789 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:32.718702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99119f-0545-4904-b8af-ad5147ee2789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:32.719106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e99119f-0545-4904-b8af-ad5147ee2789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:32.721843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99119f-0545-4904-b8af-ad5147ee2789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:32.728498 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e99119f-0545-4904-b8af-ad5147ee2789 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:32.762066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 895/1802] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:22:32.768588 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99119f-0545-4904-b8af-ad5147ee2789 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:32.790739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e99119f-0545-4904-b8af-ad5147ee2789 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:32.793526 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408afe8-be02-4f25-8d75-40ff03ec3612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:32.794648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408afe8-be02-4f25-8d75-40ff03ec3612 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:32.794997 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408afe8-be02-4f25-8d75-40ff03ec3612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:32.796316 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408afe8-be02-4f25-8d75-40ff03ec3612 admin admin] PATH_INFO: `/v3/projects/997a2bb6b781476fb26522ef6c229e44/users/54fdcaab1da94b40acf08851e4d1841c/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:32.798994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408afe8-be02-4f25-8d75-40ff03ec3612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=997a2bb6b781476fb26522ef6c229e44, user_id=54fdcaab1da94b40acf08851e4d1841c, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:32.801938 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408afe8-be02-4f25-8d75-40ff03ec3612 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:32.822032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 896/1803] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:32 2022] PUT /identity/v3/projects/997a2bb6b781476fb26522ef6c229e44/users/54fdcaab1da94b40acf08851e4d1841c/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:32.834927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99b41b-82de-4c6b-9e24-bc27d056e618 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:32.835246 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99b41b-82de-4c6b-9e24-bc27d056e618 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:32.835504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99b41b-82de-4c6b-9e24-bc27d056e618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:32.867283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a6916c55-e998-4cd3-b38c-1a0d13887c23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfc7ab38096d4dcb8c5470db63ccc7b4'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:32.895621 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a6916c55-e998-4cd3-b38c-1a0d13887c23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfc7ab38096d4dcb8c5470db63ccc7b4'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:32.899336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-2a99b41b-82de-4c6b-9e24-bc27d056e618 None None] MFA Rules not processed for user `54fdcaab1da94b40acf08851e4d1841c`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:32.900805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-2a99b41b-82de-4c6b-9e24-bc27d056e618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:32.978185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a6916c55-e998-4cd3-b38c-1a0d13887c23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfc7ab38096d4dcb8c5470db63ccc7b4'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:32.988373 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a6916c55-e998-4cd3-b38c-1a0d13887c23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfc7ab38096d4dcb8c5470db63ccc7b4'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:32.989217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a6916c55-e998-4cd3-b38c-1a0d13887c23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfc7ab38096d4dcb8c5470db63ccc7b4'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:32.996094 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 908/1804] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:32 2022] DELETE /identity/v3/users/cfc7ab38096d4dcb8c5470db63ccc7b4 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:33.006436 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 897/1805] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:33.022383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6916c55-e998-4cd3-b38c-1a0d13887c23 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:33.025460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-a6916c55-e998-4cd3-b38c-1a0d13887c23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:33.038757 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99b41b-82de-4c6b-9e24-bc27d056e618 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:33.040780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-2a99b41b-82de-4c6b-9e24-bc27d056e618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:33.232443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6916c55-e998-4cd3-b38c-1a0d13887c23 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:33.242212 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99b41b-82de-4c6b-9e24-bc27d056e618 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:33.248003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e81fb-7f90-41b3-8e94-b083d5756338 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:33.249462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e81fb-7f90-41b3-8e94-b083d5756338 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:33.250168 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e81fb-7f90-41b3-8e94-b083d5756338 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:33.255284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e81fb-7f90-41b3-8e94-b083d5756338 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:33.267751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e58e81fb-7f90-41b3-8e94-b083d5756338 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:33.409342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:33.430022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527682b1-bd87-422d-9686-2ab19870762f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:33.434009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527682b1-bd87-422d-9686-2ab19870762f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:33.434325 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527682b1-bd87-422d-9686-2ab19870762f admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:33.437494 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527682b1-bd87-422d-9686-2ab19870762f admin admin] PATH_INFO: `/v3/projects/76207cb13c7b4878954698bd438ee692` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:33.441337 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527682b1-bd87-422d-9686-2ab19870762f admin admin] RBAC: Authorizing `identity:delete_project(project_id=76207cb13c7b4878954698bd438ee692)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:33.444726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527682b1-bd87-422d-9686-2ab19870762f admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:33.516362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e81fb-7f90-41b3-8e94-b083d5756338 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:33.520067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e81fb-7f90-41b3-8e94-b083d5756338 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:33.573948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 909/1806] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:22:33.678021 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-527682b1-bd87-422d-9686-2ab19870762f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76207cb13c7b4878954698bd438ee692', 'request_id': 'req-527682b1-bd87-422d-9686-2ab19870762f'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:33.692294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-527682b1-bd87-422d-9686-2ab19870762f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76207cb13c7b4878954698bd438ee692', 'request_id': 'req-527682b1-bd87-422d-9686-2ab19870762f'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:33.766649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e81fb-7f90-41b3-8e94-b083d5756338 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:33.778402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e81fb-7f90-41b3-8e94-b083d5756338 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:33.847443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-527682b1-bd87-422d-9686-2ab19870762f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76207cb13c7b4878954698bd438ee692', 'request_id': 'req-527682b1-bd87-422d-9686-2ab19870762f'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:33.848948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd903ac-9384-4edc-adba-f7c5a2b058ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:33.849914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd903ac-9384-4edc-adba-f7c5a2b058ec service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:33.850350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd903ac-9384-4edc-adba-f7c5a2b058ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:33.850774 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd903ac-9384-4edc-adba-f7c5a2b058ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:33.853256 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-bbd903ac-9384-4edc-adba-f7c5a2b058ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:33.863063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-527682b1-bd87-422d-9686-2ab19870762f admin admin] The token cache is being invalidate because project 76207cb13c7b4878954698bd438ee692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:33.863371 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-527682b1-bd87-422d-9686-2ab19870762f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:33.864094 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-527682b1-bd87-422d-9686-2ab19870762f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:33.874634 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 898/1807] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:33 2022] DELETE /identity/v3/projects/76207cb13c7b4878954698bd438ee692 => generated 0 bytes in 828 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:33.874634 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9912ef-a011-4d4f-af64-ef5bf7aa8a7f None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:33.874634 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9912ef-a011-4d4f-af64-ef5bf7aa8a7f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:33.874634 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9912ef-a011-4d4f-af64-ef5bf7aa8a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:33.897748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:33.912426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-fb9912ef-a011-4d4f-af64-ef5bf7aa8a7f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:33.913782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-fb9912ef-a011-4d4f-af64-ef5bf7aa8a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:33.942142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd903ac-9384-4edc-adba-f7c5a2b058ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:33.943908 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd903ac-9384-4edc-adba-f7c5a2b058ec service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:33.954874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 910/1808] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:22:33.989356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 899/1809] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:33.997433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4925f52-3007-42b0-a53a-c62ec97e30db None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:33.997728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4925f52-3007-42b0-a53a-c62ec97e30db None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:33.998236 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4925f52-3007-42b0-a53a-c62ec97e30db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:34.019707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-f4925f52-3007-42b0-a53a-c62ec97e30db None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:34.020308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9912ef-a011-4d4f-af64-ef5bf7aa8a7f None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:34.020990 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f4925f52-3007-42b0-a53a-c62ec97e30db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:34.021983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-fb9912ef-a011-4d4f-af64-ef5bf7aa8a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:34.035221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 911/1810] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:34.039296 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4925f52-3007-42b0-a53a-c62ec97e30db None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:34.049931 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4925f52-3007-42b0-a53a-c62ec97e30db None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:34.056210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973f1e9-1f13-4741-af6d-2cea5d2a1332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:34.057176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973f1e9-1f13-4741-af6d-2cea5d2a1332 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:34.057472 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973f1e9-1f13-4741-af6d-2cea5d2a1332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:34.057735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973f1e9-1f13-4741-af6d-2cea5d2a1332 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:34.058490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973f1e9-1f13-4741-af6d-2cea5d2a1332 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:34.058796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973f1e9-1f13-4741-af6d-2cea5d2a1332 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:34.059441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9912ef-a011-4d4f-af64-ef5bf7aa8a7f None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:34.060412 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973f1e9-1f13-4741-af6d-2cea5d2a1332 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:34.068675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25183a4b-691b-4bb4-9710-614b705b712b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:34.070485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183a4b-691b-4bb4-9710-614b705b712b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:34.070906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183a4b-691b-4bb4-9710-614b705b712b admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:34.071264 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25183a4b-691b-4bb4-9710-614b705b712b admin admin] PATH_INFO: `/v3/projects/0eeb804e7f1a44ee9032d443b1b53ef4` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:34.072734 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25183a4b-691b-4bb4-9710-614b705b712b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eeb804e7f1a44ee9032d443b1b53ef4)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:34.076093 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25183a4b-691b-4bb4-9710-614b705b712b admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:34.078089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 912/1811] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:22:34.085011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973f1e9-1f13-4741-af6d-2cea5d2a1332 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:34.101768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973f1e9-1f13-4741-af6d-2cea5d2a1332 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:34.107992 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa796559-53f3-4528-8d9a-07aa36b20f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:34.108682 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa796559-53f3-4528-8d9a-07aa36b20f66 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:34.109001 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa796559-53f3-4528-8d9a-07aa36b20f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:34.109297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa796559-53f3-4528-8d9a-07aa36b20f66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:34.110004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa796559-53f3-4528-8d9a-07aa36b20f66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:34.112672 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa796559-53f3-4528-8d9a-07aa36b20f66 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:34.224680 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-25183a4b-691b-4bb4-9710-614b705b712b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eeb804e7f1a44ee9032d443b1b53ef4', 'request_id': 'req-25183a4b-691b-4bb4-9710-614b705b712b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:34.237363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-25183a4b-691b-4bb4-9710-614b705b712b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eeb804e7f1a44ee9032d443b1b53ef4', 'request_id': 'req-25183a4b-691b-4bb4-9710-614b705b712b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:34.247082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 913/1812] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:34 2022] POST /identity/v3/projects => generated 369 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:34.253366 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa796559-53f3-4528-8d9a-07aa36b20f66 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:34.323099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa796559-53f3-4528-8d9a-07aa36b20f66 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:34.327966 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b60512-957f-4e51-b900-0277558d3f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:34.328624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b60512-957f-4e51-b900-0277558d3f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:34.328905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b60512-957f-4e51-b900-0277558d3f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:34.329328 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b60512-957f-4e51-b900-0277558d3f7e admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:34.329959 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b60512-957f-4e51-b900-0277558d3f7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:34.330905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-25183a4b-691b-4bb4-9710-614b705b712b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eeb804e7f1a44ee9032d443b1b53ef4', 'request_id': 'req-25183a4b-691b-4bb4-9710-614b705b712b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:34.331998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b60512-957f-4e51-b900-0277558d3f7e admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:34.353296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-25183a4b-691b-4bb4-9710-614b705b712b admin admin] The token cache is being invalidate because project 0eeb804e7f1a44ee9032d443b1b53ef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:34.353487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-25183a4b-691b-4bb4-9710-614b705b712b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:34.354160 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-25183a4b-691b-4bb4-9710-614b705b712b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:34.355815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 900/1813] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:34 2022] DELETE /identity/v3/projects/0eeb804e7f1a44ee9032d443b1b53ef4 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:34.361226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447276b-d4f2-44c6-bed0-7bb63a9adc62 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:34.362062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447276b-d4f2-44c6-bed0-7bb63a9adc62 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:34.362062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447276b-d4f2-44c6-bed0-7bb63a9adc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:34.379880 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-0447276b-d4f2-44c6-bed0-7bb63a9adc62 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:34.381123 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-0447276b-d4f2-44c6-bed0-7bb63a9adc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:34.389054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 901/1814] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:34.397482 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e94c45-8a64-4453-8bbe-6117517daa0d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:34.397758 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e94c45-8a64-4453-8bbe-6117517daa0d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:34.398078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e94c45-8a64-4453-8bbe-6117517daa0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:34.418896 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-32e94c45-8a64-4453-8bbe-6117517daa0d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:34.424711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-32e94c45-8a64-4453-8bbe-6117517daa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:34.424791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 914/1815] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:34 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:34.428130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b60512-957f-4e51-b900-0277558d3f7e admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:34.431586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-57b60512-957f-4e51-b900-0277558d3f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:34.434550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 902/1816] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:34.439694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e94c45-8a64-4453-8bbe-6117517daa0d None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:34.450933 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b60512-957f-4e51-b900-0277558d3f7e admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:34.455303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e94c45-8a64-4453-8bbe-6117517daa0d None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:34.456720 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e568b-f989-4986-b271-9b07496f8488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:34.457545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e568b-f989-4986-b271-9b07496f8488 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:34.457890 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e568b-f989-4986-b271-9b07496f8488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:34.458377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e568b-f989-4986-b271-9b07496f8488 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:34.459198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e568b-f989-4986-b271-9b07496f8488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:34.460067 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5e67d-5d3d-4280-b4c1-65ab0c0fdc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:34.460791 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5e67d-5d3d-4280-b4c1-65ab0c0fdc2e admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:34.461200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5e67d-5d3d-4280-b4c1-65ab0c0fdc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:34.461400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5e67d-5d3d-4280-b4c1-65ab0c0fdc2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:34.462146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5e67d-5d3d-4280-b4c1-65ab0c0fdc2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:34.462470 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5e67d-5d3d-4280-b4c1-65ab0c0fdc2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:34.464092 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5e67d-5d3d-4280-b4c1-65ab0c0fdc2e admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:34.471608 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e568b-f989-4986-b271-9b07496f8488 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:34.478604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 903/1817] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:34.483006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5e67d-5d3d-4280-b4c1-65ab0c0fdc2e admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:34.485399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 915/1818] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:22:34.490266 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e568b-f989-4986-b271-9b07496f8488 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:34.495433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5e67d-5d3d-4280-b4c1-65ab0c0fdc2e admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:34.500824 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135eeffd-6aef-42d8-89b0-33ee02917df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:34.501042 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e568b-f989-4986-b271-9b07496f8488 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:34.502410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135eeffd-6aef-42d8-89b0-33ee02917df0 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:34.503003 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135eeffd-6aef-42d8-89b0-33ee02917df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:34.503394 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135eeffd-6aef-42d8-89b0-33ee02917df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:34.504256 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135eeffd-6aef-42d8-89b0-33ee02917df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:34.506240 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135eeffd-6aef-42d8-89b0-33ee02917df0 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:34.506381 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea184f-76db-4e01-869c-47282dd198cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:34.507031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea184f-76db-4e01-869c-47282dd198cc admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:34.507313 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea184f-76db-4e01-869c-47282dd198cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:34.507594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea184f-76db-4e01-869c-47282dd198cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:34.508340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea184f-76db-4e01-869c-47282dd198cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:34.510027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea184f-76db-4e01-869c-47282dd198cc admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:34.530327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 916/1819] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:22:34.535092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea184f-76db-4e01-869c-47282dd198cc admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:34.546208 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea184f-76db-4e01-869c-47282dd198cc admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:34.552214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff7761-355d-4ecb-8778-90682a1f5c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:34.553304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff7761-355d-4ecb-8778-90682a1f5c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:34.556766 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff7761-355d-4ecb-8778-90682a1f5c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:34.556766 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff7761-355d-4ecb-8778-90682a1f5c53 admin admin] PATH_INFO: `/v3/projects/f20b9baaa44d465194a8368fe6acbd72/users/26262c9e961745bbb349c1e8f1cc2dca/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:34.570709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff7761-355d-4ecb-8778-90682a1f5c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20b9baaa44d465194a8368fe6acbd72, user_id=26262c9e961745bbb349c1e8f1cc2dca, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:34.574742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff7761-355d-4ecb-8778-90682a1f5c53 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:34.607389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 904/1820] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:34 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:34.613257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135eeffd-6aef-42d8-89b0-33ee02917df0 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:34.664235 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 917/1821] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:34 2022] PUT /identity/v3/projects/f20b9baaa44d465194a8368fe6acbd72/users/26262c9e961745bbb349c1e8f1cc2dca/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:34.682820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71631c-cad5-4372-81cf-86ca4929c539 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:34.683115 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71631c-cad5-4372-81cf-86ca4929c539 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:34.683397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71631c-cad5-4372-81cf-86ca4929c539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:34.691294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135eeffd-6aef-42d8-89b0-33ee02917df0 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:34.729849 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-ae71631c-cad5-4372-81cf-86ca4929c539 None None] MFA Rules not processed for user `26262c9e961745bbb349c1e8f1cc2dca`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:34.731147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ae71631c-cad5-4372-81cf-86ca4929c539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:34.767917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afe6a8-d5e2-4aa3-b1b2-f262b5c8fc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:34.768944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afe6a8-d5e2-4aa3-b1b2-f262b5c8fc22 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:34.769285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afe6a8-d5e2-4aa3-b1b2-f262b5c8fc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:34.769745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afe6a8-d5e2-4aa3-b1b2-f262b5c8fc22 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:34.770619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9afe6a8-d5e2-4aa3-b1b2-f262b5c8fc22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:34.772581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9afe6a8-d5e2-4aa3-b1b2-f262b5c8fc22 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:34.809280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 918/1822] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:34.816977 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71631c-cad5-4372-81cf-86ca4929c539 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:34.830204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71631c-cad5-4372-81cf-86ca4929c539 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:34.835052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ab337-11b5-4107-8ec4-ed3b04dda1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:34.835729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ab337-11b5-4107-8ec4-ed3b04dda1f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:34.836044 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ab337-11b5-4107-8ec4-ed3b04dda1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:34.836425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ab337-11b5-4107-8ec4-ed3b04dda1f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:34.837160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ab337-11b5-4107-8ec4-ed3b04dda1f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:34.839073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ab337-11b5-4107-8ec4-ed3b04dda1f3 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:35.000380 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 919/1823] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:34 2022] POST /identity/v3/projects => generated 369 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:35.000380 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ab337-11b5-4107-8ec4-ed3b04dda1f3 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:35.001435 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 905/1824] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:34 2022] POST /identity/v3/users => generated 293 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:35.003861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afe6a8-d5e2-4aa3-b1b2-f262b5c8fc22 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:35.015050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afe6a8-d5e2-4aa3-b1b2-f262b5c8fc22 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:35.062621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:35.082188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ab337-11b5-4107-8ec4-ed3b04dda1f3 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:35.085395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edee8f-9075-462b-916d-e8c9341d43d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:35.086100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edee8f-9075-462b-916d-e8c9341d43d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:35.086389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edee8f-9075-462b-916d-e8c9341d43d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:35.086669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edee8f-9075-462b-916d-e8c9341d43d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:35.087380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edee8f-9075-462b-916d-e8c9341d43d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:35.089366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edee8f-9075-462b-916d-e8c9341d43d2 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:35.089769 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94397aeb-9593-4a2e-9808-1fce7db98db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:35.090748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94397aeb-9593-4a2e-9808-1fce7db98db3 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:35.091205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94397aeb-9593-4a2e-9808-1fce7db98db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:35.091653 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94397aeb-9593-4a2e-9808-1fce7db98db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:35.092637 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94397aeb-9593-4a2e-9808-1fce7db98db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:35.094708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94397aeb-9593-4a2e-9808-1fce7db98db3 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:35.108522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 920/1825] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:22:35.121177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94397aeb-9593-4a2e-9808-1fce7db98db3 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:35.127591 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94397aeb-9593-4a2e-9808-1fce7db98db3 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:35.132420 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee06b70-241c-4268-96cc-accce82a3e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:35.133151 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee06b70-241c-4268-96cc-accce82a3e3b admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:35.133538 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee06b70-241c-4268-96cc-accce82a3e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:35.133722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee06b70-241c-4268-96cc-accce82a3e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:35.134448 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee06b70-241c-4268-96cc-accce82a3e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:35.136372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee06b70-241c-4268-96cc-accce82a3e3b admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:35.147397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 921/1826] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:35.155138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee06b70-241c-4268-96cc-accce82a3e3b admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:35.166893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee06b70-241c-4268-96cc-accce82a3e3b admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:35.172786 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea770bc7-3488-4a06-b13e-52c92b5d0191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:35.174660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea770bc7-3488-4a06-b13e-52c92b5d0191 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:35.174864 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea770bc7-3488-4a06-b13e-52c92b5d0191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:35.175263 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea770bc7-3488-4a06-b13e-52c92b5d0191 admin admin] PATH_INFO: `/v3/projects/7cae8db77a4d4647b349d0bd4dd056ba/users/0c08efdef298498592a77dfe5e85f717/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:35.193725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea770bc7-3488-4a06-b13e-52c92b5d0191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cae8db77a4d4647b349d0bd4dd056ba, user_id=0c08efdef298498592a77dfe5e85f717, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:35.197078 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea770bc7-3488-4a06-b13e-52c92b5d0191 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:35.221378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:35.229036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 906/1827] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:35 2022] POST /identity/v3/users => generated 291 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:35.230870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edee8f-9075-462b-916d-e8c9341d43d2 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:35.242280 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edee8f-9075-462b-916d-e8c9341d43d2 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:35.247844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d8947-78a9-494c-a20a-d5917276dc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:35.248530 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d8947-78a9-494c-a20a-d5917276dc02 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:35.248836 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d8947-78a9-494c-a20a-d5917276dc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:35.249181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d8947-78a9-494c-a20a-d5917276dc02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:35.252661 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d8947-78a9-494c-a20a-d5917276dc02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:35.252661 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d8947-78a9-494c-a20a-d5917276dc02 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:35.267907 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 907/1828] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:22:35.269203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d8947-78a9-494c-a20a-d5917276dc02 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:35.277688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 922/1829] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:35 2022] PUT /identity/v3/projects/7cae8db77a4d4647b349d0bd4dd056ba/users/0c08efdef298498592a77dfe5e85f717/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:35.288606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d8947-78a9-494c-a20a-d5917276dc02 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:35.300896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c6a80-3a57-4709-b9e8-d0835fba4dcc None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:35.301216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c6a80-3a57-4709-b9e8-d0835fba4dcc None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:35.301496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c6a80-3a57-4709-b9e8-d0835fba4dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:35.333063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-4c2c6a80-3a57-4709-b9e8-d0835fba4dcc None None] MFA Rules not processed for user `0c08efdef298498592a77dfe5e85f717`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:35.335233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-4c2c6a80-3a57-4709-b9e8-d0835fba4dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:35.362251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e50375-e48f-49e5-bcb0-d685e32f3f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:35.363172 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e50375-e48f-49e5-bcb0-d685e32f3f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:35.363468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e50375-e48f-49e5-bcb0-d685e32f3f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:35.363738 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e50375-e48f-49e5-bcb0-d685e32f3f67 admin admin] PATH_INFO: `/v3/projects/a51add7be5884f53b41c7017fc2d5970/users/11e03f13344c495881baf85fe15763ab/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:35.382416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e50375-e48f-49e5-bcb0-d685e32f3f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a51add7be5884f53b41c7017fc2d5970, user_id=11e03f13344c495881baf85fe15763ab, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:35.385309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e50375-e48f-49e5-bcb0-d685e32f3f67 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:35.408665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 923/1830] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:35.420742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c6a80-3a57-4709-b9e8-d0835fba4dcc None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:35.422531 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-4c2c6a80-3a57-4709-b9e8-d0835fba4dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:35.481117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 908/1831] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:35 2022] PUT /identity/v3/projects/a51add7be5884f53b41c7017fc2d5970/users/11e03f13344c495881baf85fe15763ab/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:35.485226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e50375-e48f-49e5-bcb0-d685e32f3f67 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:35.490689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c6a80-3a57-4709-b9e8-d0835fba4dcc None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:35.555775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56157d-c963-4685-abcf-146b7efc5157 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:35.556057 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56157d-c963-4685-abcf-146b7efc5157 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:35.556341 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56157d-c963-4685-abcf-146b7efc5157 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:35.556620 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56157d-c963-4685-abcf-146b7efc5157 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:35.582838 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e50375-e48f-49e5-bcb0-d685e32f3f67 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:35.590475 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4982026-86dd-4a14-b97c-6f79e742e2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:35.591817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4982026-86dd-4a14-b97c-6f79e742e2e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:35.592319 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4982026-86dd-4a14-b97c-6f79e742e2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:35.592639 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4982026-86dd-4a14-b97c-6f79e742e2e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:35.603682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4982026-86dd-4a14-b97c-6f79e742e2e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:35.609881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4982026-86dd-4a14-b97c-6f79e742e2e3 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:35.631734 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 909/1832] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:35.638214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4982026-86dd-4a14-b97c-6f79e742e2e3 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:35.659048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4982026-86dd-4a14-b97c-6f79e742e2e3 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:35.666491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606ab23-2619-431b-b755-569f379ed645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:35.667777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606ab23-2619-431b-b755-569f379ed645 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:35.668311 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606ab23-2619-431b-b755-569f379ed645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:35.669100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606ab23-2619-431b-b755-569f379ed645 admin admin] PATH_INFO: `/v3/projects/a51add7be5884f53b41c7017fc2d5970/users/11e03f13344c495881baf85fe15763ab/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:35.699104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606ab23-2619-431b-b755-569f379ed645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a51add7be5884f53b41c7017fc2d5970, user_id=11e03f13344c495881baf85fe15763ab, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:35.701029 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56157d-c963-4685-abcf-146b7efc5157 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:35.702471 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606ab23-2619-431b-b755-569f379ed645 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:35.704133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56157d-c963-4685-abcf-146b7efc5157 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:35.718128 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 910/1833] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:35 2022] PUT /identity/v3/projects/a51add7be5884f53b41c7017fc2d5970/users/11e03f13344c495881baf85fe15763ab/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:35.732581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f7fe4-5508-4e4e-ba3b-d04c70077dae None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:35.732898 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f7fe4-5508-4e4e-ba3b-d04c70077dae None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:35.737514 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f7fe4-5508-4e4e-ba3b-d04c70077dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:35.764005 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 924/1834] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:22:35.796791 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-ee3f7fe4-5508-4e4e-ba3b-d04c70077dae None None] MFA Rules not processed for user `11e03f13344c495881baf85fe15763ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:35.799170 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ee3f7fe4-5508-4e4e-ba3b-d04c70077dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:35.972790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 911/1835] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:35 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:35.993042 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56157d-c963-4685-abcf-146b7efc5157 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:35.995658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9a56157d-c963-4685-abcf-146b7efc5157 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:36.010353 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56157d-c963-4685-abcf-146b7efc5157 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:36.160029 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb63c88-5867-4de1-9fe4-d9aefe705efb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:36.161415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb63c88-5867-4de1-9fe4-d9aefe705efb service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:36.162936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb63c88-5867-4de1-9fe4-d9aefe705efb service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:36.163303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb63c88-5867-4de1-9fe4-d9aefe705efb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:36.195778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb63c88-5867-4de1-9fe4-d9aefe705efb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:36.197028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb63c88-5867-4de1-9fe4-d9aefe705efb service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:36.219585 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 925/1836] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:35 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:36.381791 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f7fe4-5508-4e4e-ba3b-d04c70077dae None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:36.393307 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f7fe4-5508-4e4e-ba3b-d04c70077dae None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:36.398433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78faa8-89dc-4c26-8ab9-ac341b7e4514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:36.399331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78faa8-89dc-4c26-8ab9-ac341b7e4514 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:36.399745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78faa8-89dc-4c26-8ab9-ac341b7e4514 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:36.400161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78faa8-89dc-4c26-8ab9-ac341b7e4514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:36.489891 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78faa8-89dc-4c26-8ab9-ac341b7e4514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:36.491649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78faa8-89dc-4c26-8ab9-ac341b7e4514 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:36.509106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 912/1837] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:36 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:36.654846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:37.154526 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:38.004810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb63c88-5867-4de1-9fe4-d9aefe705efb service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:38.006463 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-afb63c88-5867-4de1-9fe4-d9aefe705efb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:38.092053 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb63c88-5867-4de1-9fe4-d9aefe705efb service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:38.097872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0297b-6df1-4505-adab-1cb0389a1ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:38.100728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0297b-6df1-4505-adab-1cb0389a1ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:38.100728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0297b-6df1-4505-adab-1cb0389a1ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:38.100728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0297b-6df1-4505-adab-1cb0389a1ac0 admin admin] PATH_INFO: `/v3/users/384b9998ce4945d3879362c03723cc3b` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:38.100728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0297b-6df1-4505-adab-1cb0389a1ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=384b9998ce4945d3879362c03723cc3b)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:38.102425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0297b-6df1-4505-adab-1cb0389a1ac0 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:38.303827 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-daf0297b-6df1-4505-adab-1cb0389a1ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '384b9998ce4945d3879362c03723cc3b'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:38.318502 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-daf0297b-6df1-4505-adab-1cb0389a1ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '384b9998ce4945d3879362c03723cc3b'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:38.385899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-daf0297b-6df1-4505-adab-1cb0389a1ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '384b9998ce4945d3879362c03723cc3b'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:38.394253 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-daf0297b-6df1-4505-adab-1cb0389a1ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '384b9998ce4945d3879362c03723cc3b'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:38.394963 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-daf0297b-6df1-4505-adab-1cb0389a1ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '384b9998ce4945d3879362c03723cc3b'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:38.401837 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 926/1838] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:38 2022] DELETE /identity/v3/users/384b9998ce4945d3879362c03723cc3b => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:38.406539 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbe42b-c03d-4467-8204-5275f0dbd2c7 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:38.407000 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbe42b-c03d-4467-8204-5275f0dbd2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:38.407087 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbe42b-c03d-4467-8204-5275f0dbd2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:38.428128 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-9cdbe42b-c03d-4467-8204-5275f0dbd2c7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:38.429494 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9cdbe42b-c03d-4467-8204-5275f0dbd2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:38.505385 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 913/1839] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:38.514627 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0297b-6df1-4505-adab-1cb0389a1ac0 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:38.516193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-daf0297b-6df1-4505-adab-1cb0389a1ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:38.582417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0297b-6df1-4505-adab-1cb0389a1ac0 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:38.588342 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cbe4ec-cfa4-4eeb-acac-a261aefd6545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:38.591358 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbe4ec-cfa4-4eeb-acac-a261aefd6545 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:38.591358 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbe4ec-cfa4-4eeb-acac-a261aefd6545 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:38.591358 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbe4ec-cfa4-4eeb-acac-a261aefd6545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:38.600367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cbe4ec-cfa4-4eeb-acac-a261aefd6545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:38.601936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cbe4ec-cfa4-4eeb-acac-a261aefd6545 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:38.613683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 927/1840] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:38.982147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:39.351132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbe42b-c03d-4467-8204-5275f0dbd2c7 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:39.352654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9cdbe42b-c03d-4467-8204-5275f0dbd2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:39.366452 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbe42b-c03d-4467-8204-5275f0dbd2c7 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:39.371747 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0e9c1-1940-4ea3-832d-111479a95ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:39.372669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0e9c1-1940-4ea3-832d-111479a95ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:39.372976 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0e9c1-1940-4ea3-832d-111479a95ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:39.373799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0e9c1-1940-4ea3-832d-111479a95ab9 admin admin] PATH_INFO: `/v3/projects/e4a070b8ebca4ed096879ff5077f8272` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:39.375106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0e9c1-1940-4ea3-832d-111479a95ab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4a070b8ebca4ed096879ff5077f8272)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:39.382840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0e9c1-1940-4ea3-832d-111479a95ab9 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:39.428028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:39.495023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c4c0e9c1-1940-4ea3-832d-111479a95ab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4a070b8ebca4ed096879ff5077f8272', 'request_id': 'req-c4c0e9c1-1940-4ea3-832d-111479a95ab9'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:39.502506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c4c0e9c1-1940-4ea3-832d-111479a95ab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4a070b8ebca4ed096879ff5077f8272', 'request_id': 'req-c4c0e9c1-1940-4ea3-832d-111479a95ab9'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:39.582206 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c4c0e9c1-1940-4ea3-832d-111479a95ab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4a070b8ebca4ed096879ff5077f8272', 'request_id': 'req-c4c0e9c1-1940-4ea3-832d-111479a95ab9'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:39.594412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c4c0e9c1-1940-4ea3-832d-111479a95ab9 admin admin] The token cache is being invalidate because project e4a070b8ebca4ed096879ff5077f8272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:39.595464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c4c0e9c1-1940-4ea3-832d-111479a95ab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:39.596377 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-c4c0e9c1-1940-4ea3-832d-111479a95ab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:39.598050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 914/1841] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:39 2022] DELETE /identity/v3/projects/e4a070b8ebca4ed096879ff5077f8272 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:39.602202 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cbe4ec-cfa4-4eeb-acac-a261aefd6545 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:39.613821 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cbe4ec-cfa4-4eeb-acac-a261aefd6545 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:39.674995 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79142f-ed7f-4963-9859-c28eaf7567b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:39.675927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79142f-ed7f-4963-9859-c28eaf7567b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:39.676217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79142f-ed7f-4963-9859-c28eaf7567b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:39.676525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79142f-ed7f-4963-9859-c28eaf7567b2 admin admin] PATH_INFO: `/v3/users/54fdcaab1da94b40acf08851e4d1841c` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:39.677832 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79142f-ed7f-4963-9859-c28eaf7567b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54fdcaab1da94b40acf08851e4d1841c)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:39.679819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79142f-ed7f-4963-9859-c28eaf7567b2 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:39.901619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a79142f-ed7f-4963-9859-c28eaf7567b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54fdcaab1da94b40acf08851e4d1841c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:39.913999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a79142f-ed7f-4963-9859-c28eaf7567b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54fdcaab1da94b40acf08851e4d1841c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:40.011179 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a79142f-ed7f-4963-9859-c28eaf7567b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54fdcaab1da94b40acf08851e4d1841c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:40.019538 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a79142f-ed7f-4963-9859-c28eaf7567b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54fdcaab1da94b40acf08851e4d1841c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:40.020298 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a79142f-ed7f-4963-9859-c28eaf7567b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54fdcaab1da94b40acf08851e4d1841c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:40.026713 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 928/1842] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:39 2022] DELETE /identity/v3/users/54fdcaab1da94b40acf08851e4d1841c => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:40.610087 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0e9c1-1940-4ea3-832d-111479a95ab9 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:40.612465 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c4c0e9c1-1940-4ea3-832d-111479a95ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:40.625781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0e9c1-1940-4ea3-832d-111479a95ab9 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:40.686695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8700f43-2614-4de8-aa30-801386fece86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:40.687674 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8700f43-2614-4de8-aa30-801386fece86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:40.687854 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8700f43-2614-4de8-aa30-801386fece86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:40.688115 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8700f43-2614-4de8-aa30-801386fece86 admin admin] PATH_INFO: `/v3/projects/997a2bb6b781476fb26522ef6c229e44` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:40.689983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8700f43-2614-4de8-aa30-801386fece86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=997a2bb6b781476fb26522ef6c229e44)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:40.692162 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8700f43-2614-4de8-aa30-801386fece86 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:40.867744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b8700f43-2614-4de8-aa30-801386fece86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '997a2bb6b781476fb26522ef6c229e44', 'request_id': 'req-b8700f43-2614-4de8-aa30-801386fece86'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:40.874409 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b8700f43-2614-4de8-aa30-801386fece86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '997a2bb6b781476fb26522ef6c229e44', 'request_id': 'req-b8700f43-2614-4de8-aa30-801386fece86'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:40.934540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b8700f43-2614-4de8-aa30-801386fece86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '997a2bb6b781476fb26522ef6c229e44', 'request_id': 'req-b8700f43-2614-4de8-aa30-801386fece86'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:40.949262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b8700f43-2614-4de8-aa30-801386fece86 admin admin] The token cache is being invalidate because project 997a2bb6b781476fb26522ef6c229e44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:40.949550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b8700f43-2614-4de8-aa30-801386fece86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:40.950391 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b8700f43-2614-4de8-aa30-801386fece86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:40.952185 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 915/1843] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:40 2022] DELETE /identity/v3/projects/997a2bb6b781476fb26522ef6c229e44 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:40.957350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f8240-ff18-4e98-ac6c-a7d9411d7774 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:40.957857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f8240-ff18-4e98-ac6c-a7d9411d7774 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:40.957947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f8240-ff18-4e98-ac6c-a7d9411d7774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:40.976785 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-586f8240-ff18-4e98-ac6c-a7d9411d7774 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:40.978043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-586f8240-ff18-4e98-ac6c-a7d9411d7774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:41.045439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 929/1844] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:41.054087 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d9f8c-8fed-4754-a9af-b23221003165 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:41.054596 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d9f8c-8fed-4754-a9af-b23221003165 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:41.054879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d9f8c-8fed-4754-a9af-b23221003165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:41.064488 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:41.076423 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-1c3d9f8c-8fed-4754-a9af-b23221003165 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:41.077767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1c3d9f8c-8fed-4754-a9af-b23221003165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:41.087631 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 916/1845] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:41.091972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f8240-ff18-4e98-ac6c-a7d9411d7774 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:41.102675 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f8240-ff18-4e98-ac6c-a7d9411d7774 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:41.107417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202893d-cad7-4ce0-ad73-8d48c43132f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:41.108082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202893d-cad7-4ce0-ad73-8d48c43132f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:41.108365 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202893d-cad7-4ce0-ad73-8d48c43132f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:41.108641 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202893d-cad7-4ce0-ad73-8d48c43132f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:41.109401 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0202893d-cad7-4ce0-ad73-8d48c43132f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:41.109703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0202893d-cad7-4ce0-ad73-8d48c43132f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:41.111989 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0202893d-cad7-4ce0-ad73-8d48c43132f5 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:41.124557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 930/1846] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:41.129558 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d9f8c-8fed-4754-a9af-b23221003165 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:41.139205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d9f8c-8fed-4754-a9af-b23221003165 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:41.144328 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb55fe7-27c6-4953-a057-0eb5b2ea38d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:41.145130 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb55fe7-27c6-4953-a057-0eb5b2ea38d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:41.145443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb55fe7-27c6-4953-a057-0eb5b2ea38d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:41.145721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb55fe7-27c6-4953-a057-0eb5b2ea38d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:41.146428 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb55fe7-27c6-4953-a057-0eb5b2ea38d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:41.148794 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb55fe7-27c6-4953-a057-0eb5b2ea38d5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:41.189202 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:41.278861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 917/1847] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:41 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:41.283973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202893d-cad7-4ce0-ad73-8d48c43132f5 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:41.294343 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202893d-cad7-4ce0-ad73-8d48c43132f5 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:41.358545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e869399-de6e-4c94-b410-caa9ab9ff11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:41.359668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e869399-de6e-4c94-b410-caa9ab9ff11e admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:41.360099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e869399-de6e-4c94-b410-caa9ab9ff11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:41.361090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e869399-de6e-4c94-b410-caa9ab9ff11e admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:41.362028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e869399-de6e-4c94-b410-caa9ab9ff11e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:41.364981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e869399-de6e-4c94-b410-caa9ab9ff11e admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:41.475830 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 931/1848] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:41 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:41.477203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb55fe7-27c6-4953-a057-0eb5b2ea38d5 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:41.488571 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb55fe7-27c6-4953-a057-0eb5b2ea38d5 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:41.494611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b27542-81e2-4ef7-bc96-d199085840ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:41.495454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b27542-81e2-4ef7-bc96-d199085840ce admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:41.503649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b27542-81e2-4ef7-bc96-d199085840ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:41.503649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b27542-81e2-4ef7-bc96-d199085840ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:41.503649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b27542-81e2-4ef7-bc96-d199085840ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:41.503649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b27542-81e2-4ef7-bc96-d199085840ce admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:41.513223 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 918/1849] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:41.518852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e869399-de6e-4c94-b410-caa9ab9ff11e admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:41.529778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e869399-de6e-4c94-b410-caa9ab9ff11e admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:41.542086 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15aac2-5fcf-4553-ab9a-e6c7d7d5bbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:41.543894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15aac2-5fcf-4553-ab9a-e6c7d7d5bbab admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:41.545616 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15aac2-5fcf-4553-ab9a-e6c7d7d5bbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:41.546217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15aac2-5fcf-4553-ab9a-e6c7d7d5bbab admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:41.547055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15aac2-5fcf-4553-ab9a-e6c7d7d5bbab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:41.548946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15aac2-5fcf-4553-ab9a-e6c7d7d5bbab admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:41.567064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 932/1850] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:41.567876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b27542-81e2-4ef7-bc96-d199085840ce admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:41.588125 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b27542-81e2-4ef7-bc96-d199085840ce admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:41.593726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ff93f-f025-44ea-bfce-2199a065efdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:41.595215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ff93f-f025-44ea-bfce-2199a065efdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:41.597288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ff93f-f025-44ea-bfce-2199a065efdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:41.597288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ff93f-f025-44ea-bfce-2199a065efdf admin admin] PATH_INFO: `/v3/projects/58d527df216e44dfa1a89fca94d8fda3/users/ea7ab7126db3408ab2858f6fcb3d4f20/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:41.623482 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ff93f-f025-44ea-bfce-2199a065efdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d527df216e44dfa1a89fca94d8fda3, user_id=ea7ab7126db3408ab2858f6fcb3d4f20, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:41.626728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ff93f-f025-44ea-bfce-2199a065efdf admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:41.740095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 919/1851] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:41 2022] PUT /identity/v3/projects/58d527df216e44dfa1a89fca94d8fda3/users/ea7ab7126db3408ab2858f6fcb3d4f20/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:41.750298 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932736ca-c89a-4e80-b194-07dc73ad550f None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:41.750595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932736ca-c89a-4e80-b194-07dc73ad550f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:41.751524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932736ca-c89a-4e80-b194-07dc73ad550f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:41.779518 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ff93f-f025-44ea-bfce-2199a065efdf admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:41.781473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-194ff93f-f025-44ea-bfce-2199a065efdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:41.782138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-932736ca-c89a-4e80-b194-07dc73ad550f None None] MFA Rules not processed for user `ea7ab7126db3408ab2858f6fcb3d4f20`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:41.783346 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-932736ca-c89a-4e80-b194-07dc73ad550f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:41.849717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ff93f-f025-44ea-bfce-2199a065efdf admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:41.852368 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 933/1852] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:41.854657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658bbfde-5b42-4557-b3e0-c9e875161326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:41.855549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bbfde-5b42-4557-b3e0-c9e875161326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:41.856029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bbfde-5b42-4557-b3e0-c9e875161326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:41.856157 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bbfde-5b42-4557-b3e0-c9e875161326 admin admin] PATH_INFO: `/v3/users/26262c9e961745bbb349c1e8f1cc2dca` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:41.856918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932736ca-c89a-4e80-b194-07dc73ad550f None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:41.857361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658bbfde-5b42-4557-b3e0-c9e875161326 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26262c9e961745bbb349c1e8f1cc2dca)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:41.859396 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658bbfde-5b42-4557-b3e0-c9e875161326 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:41.868528 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932736ca-c89a-4e80-b194-07dc73ad550f None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:41.873707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1d34a-03bb-4848-b998-985c4d05d6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:41.874299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1d34a-03bb-4848-b998-985c4d05d6e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:41.874584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1d34a-03bb-4848-b998-985c4d05d6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:41.874867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1d34a-03bb-4848-b998-985c4d05d6e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:41.875587 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1d34a-03bb-4848-b998-985c4d05d6e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:41.878110 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c1d34a-03bb-4848-b998-985c4d05d6e7 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:41.984664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 934/1853] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:41 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:22:41.989265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1d34a-03bb-4848-b998-985c4d05d6e7 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:42.059069 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1d34a-03bb-4848-b998-985c4d05d6e7 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:42.065205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44940ec-39d2-4560-948b-1cf87c156f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:42.066049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44940ec-39d2-4560-948b-1cf87c156f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:42.066294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-658bbfde-5b42-4557-b3e0-c9e875161326 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26262c9e961745bbb349c1e8f1cc2dca'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:42.066684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44940ec-39d2-4560-948b-1cf87c156f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:42.067053 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44940ec-39d2-4560-948b-1cf87c156f58 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:42.067857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44940ec-39d2-4560-948b-1cf87c156f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:42.070123 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44940ec-39d2-4560-948b-1cf87c156f58 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:42.168166 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-658bbfde-5b42-4557-b3e0-c9e875161326 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26262c9e961745bbb349c1e8f1cc2dca'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:42.168739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-658bbfde-5b42-4557-b3e0-c9e875161326 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26262c9e961745bbb349c1e8f1cc2dca'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:42.169362 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-658bbfde-5b42-4557-b3e0-c9e875161326 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26262c9e961745bbb349c1e8f1cc2dca'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:42.177363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 935/1854] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:41 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:22:42.182906 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44940ec-39d2-4560-948b-1cf87c156f58 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:42.186388 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b44940ec-39d2-4560-948b-1cf87c156f58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:42.189466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-658bbfde-5b42-4557-b3e0-c9e875161326 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26262c9e961745bbb349c1e8f1cc2dca'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:42.208002 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44940ec-39d2-4560-948b-1cf87c156f58 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:42.211317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 920/1855] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:41 2022] DELETE /identity/v3/users/26262c9e961745bbb349c1e8f1cc2dca => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:42.216113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4908a4a-7883-48b4-8c27-cf1ac2205add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:42.216113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4908a4a-7883-48b4-8c27-cf1ac2205add admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:42.216113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4908a4a-7883-48b4-8c27-cf1ac2205add admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:42.216587 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4908a4a-7883-48b4-8c27-cf1ac2205add admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:42.217638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55a101-5aae-48ad-a8f5-a502e7235ccb None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:42.217781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4908a4a-7883-48b4-8c27-cf1ac2205add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:42.218112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55a101-5aae-48ad-a8f5-a502e7235ccb None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:42.218552 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb55a101-5aae-48ad-a8f5-a502e7235ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:42.221556 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4908a4a-7883-48b4-8c27-cf1ac2205add admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:42.242231 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 936/1856] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:42.248609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4908a4a-7883-48b4-8c27-cf1ac2205add admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:42.252887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-bb55a101-5aae-48ad-a8f5-a502e7235ccb None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:42.254904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-bb55a101-5aae-48ad-a8f5-a502e7235ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:42.266417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4908a4a-7883-48b4-8c27-cf1ac2205add admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:42.331266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 921/1857] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:42.340278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:INSERT=1,keystone:SELECT=97,keystone:DELETE=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:42.341588 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e0fb4-fc32-444e-a53a-5a70203b838b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:42.342276 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e0fb4-fc32-444e-a53a-5a70203b838b admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:42.342565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e0fb4-fc32-444e-a53a-5a70203b838b admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:42.342849 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e0fb4-fc32-444e-a53a-5a70203b838b admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:42.343594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e0fb4-fc32-444e-a53a-5a70203b838b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:42.345602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e0fb4-fc32-444e-a53a-5a70203b838b admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:42.348708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55a101-5aae-48ad-a8f5-a502e7235ccb None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:42.350377 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-bb55a101-5aae-48ad-a8f5-a502e7235ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:42.356898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 937/1858] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:22:42.362530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e0fb4-fc32-444e-a53a-5a70203b838b admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:42.375067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e0fb4-fc32-444e-a53a-5a70203b838b admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:42.379166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:42.383009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23303c2-dd26-4b52-a80c-2c0c47ce5035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:42.388254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23303c2-dd26-4b52-a80c-2c0c47ce5035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:42.388254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23303c2-dd26-4b52-a80c-2c0c47ce5035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:42.388254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23303c2-dd26-4b52-a80c-2c0c47ce5035 admin admin] PATH_INFO: `/v3/projects/1a594c9158a44ce4af323ba690b544c5/users/842c30e419334f769592f9d1382e78cf/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:42.406497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23303c2-dd26-4b52-a80c-2c0c47ce5035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a594c9158a44ce4af323ba690b544c5, user_id=842c30e419334f769592f9d1382e78cf, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:42.410494 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23303c2-dd26-4b52-a80c-2c0c47ce5035 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:42.410945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb55a101-5aae-48ad-a8f5-a502e7235ccb None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:42.415227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b60aea-5293-4dbc-b393-c48311805090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:42.415940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b60aea-5293-4dbc-b393-c48311805090 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:42.416232 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b60aea-5293-4dbc-b393-c48311805090 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:42.431681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b60aea-5293-4dbc-b393-c48311805090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:42.431681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b60aea-5293-4dbc-b393-c48311805090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:42.431681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b60aea-5293-4dbc-b393-c48311805090 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:42.439594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 922/1859] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:22:42.635367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 938/1860] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:42 2022] PUT /identity/v3/projects/1a594c9158a44ce4af323ba690b544c5/users/842c30e419334f769592f9d1382e78cf/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:42.638949 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e183b-dc04-4b34-9e89-c0705062dbe8 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:42.639423 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e183b-dc04-4b34-9e89-c0705062dbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:42.639716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e183b-dc04-4b34-9e89-c0705062dbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:42.690145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-202e183b-dc04-4b34-9e89-c0705062dbe8 None None] MFA Rules not processed for user `842c30e419334f769592f9d1382e78cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:42.691730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-202e183b-dc04-4b34-9e89-c0705062dbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:42.823265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 923/1861] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:42.824251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23303c2-dd26-4b52-a80c-2c0c47ce5035 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:42.906282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23303c2-dd26-4b52-a80c-2c0c47ce5035 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:42.912526 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f70a88-d1f8-4fc3-8177-3ec193bd8559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:42.913258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f70a88-d1f8-4fc3-8177-3ec193bd8559 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:42.913696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f70a88-d1f8-4fc3-8177-3ec193bd8559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:42.914153 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f70a88-d1f8-4fc3-8177-3ec193bd8559 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:42.914887 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f70a88-d1f8-4fc3-8177-3ec193bd8559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:42.916837 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f70a88-d1f8-4fc3-8177-3ec193bd8559 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:43.038135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e183b-dc04-4b34-9e89-c0705062dbe8 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:43.047214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 939/1862] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:42 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:22:43.064232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f70a88-d1f8-4fc3-8177-3ec193bd8559 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:43.071617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f8f70a88-d1f8-4fc3-8177-3ec193bd8559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:43.072922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e183b-dc04-4b34-9e89-c0705062dbe8 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:43.178120 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b63aa8-98e7-4217-a234-c7b02ed00dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:43.179363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b63aa8-98e7-4217-a234-c7b02ed00dad admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:43.179363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b63aa8-98e7-4217-a234-c7b02ed00dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:43.179363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b63aa8-98e7-4217-a234-c7b02ed00dad admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:43.181906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b63aa8-98e7-4217-a234-c7b02ed00dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:43.182054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b63aa8-98e7-4217-a234-c7b02ed00dad admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:43.205310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f70a88-d1f8-4fc3-8177-3ec193bd8559 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:43.210071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965531c-ea25-42ca-8cc2-5aacd1efe281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:43.211026 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965531c-ea25-42ca-8cc2-5aacd1efe281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:43.211448 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965531c-ea25-42ca-8cc2-5aacd1efe281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:43.211733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965531c-ea25-42ca-8cc2-5aacd1efe281 admin admin] PATH_INFO: `/v3/projects/f20b9baaa44d465194a8368fe6acbd72` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:43.212934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965531c-ea25-42ca-8cc2-5aacd1efe281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f20b9baaa44d465194a8368fe6acbd72)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:43.215012 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965531c-ea25-42ca-8cc2-5aacd1efe281 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:43.269280 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 924/1863] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:43 2022] POST /identity/v3/users => generated 295 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:22:43.273467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b63aa8-98e7-4217-a234-c7b02ed00dad admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:43.283521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b63aa8-98e7-4217-a234-c7b02ed00dad admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:43.288191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4da559-eebf-4abf-809a-26d09e054142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:43.288860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4da559-eebf-4abf-809a-26d09e054142 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:43.289203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4da559-eebf-4abf-809a-26d09e054142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:43.289487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4da559-eebf-4abf-809a-26d09e054142 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:43.290227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4da559-eebf-4abf-809a-26d09e054142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:43.291866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4da559-eebf-4abf-809a-26d09e054142 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:43.302068 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 925/1864] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:22:43.306330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4da559-eebf-4abf-809a-26d09e054142 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:43.318305 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4da559-eebf-4abf-809a-26d09e054142 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:43.323005 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a7413-1879-49ae-aa69-fa850464ee5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:43.323922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a7413-1879-49ae-aa69-fa850464ee5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:43.324230 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a7413-1879-49ae-aa69-fa850464ee5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:43.324493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a7413-1879-49ae-aa69-fa850464ee5a admin admin] PATH_INFO: `/v3/projects/cda77967154e49b6aeb6f90e64a3e31c/users/18a90c9c793a4f1687f46d082c6d96ae/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:43.346816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a7413-1879-49ae-aa69-fa850464ee5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda77967154e49b6aeb6f90e64a3e31c, user_id=18a90c9c793a4f1687f46d082c6d96ae, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:43.350388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a7413-1879-49ae-aa69-fa850464ee5a admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:43.356734 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2965531c-ea25-42ca-8cc2-5aacd1efe281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f20b9baaa44d465194a8368fe6acbd72', 'request_id': 'req-2965531c-ea25-42ca-8cc2-5aacd1efe281'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:43.363982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2965531c-ea25-42ca-8cc2-5aacd1efe281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f20b9baaa44d465194a8368fe6acbd72', 'request_id': 'req-2965531c-ea25-42ca-8cc2-5aacd1efe281'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:43.369846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2965531c-ea25-42ca-8cc2-5aacd1efe281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f20b9baaa44d465194a8368fe6acbd72', 'request_id': 'req-2965531c-ea25-42ca-8cc2-5aacd1efe281'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:43.431512 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:43.431964 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 926/1865] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:43 2022] PUT /identity/v3/projects/cda77967154e49b6aeb6f90e64a3e31c/users/18a90c9c793a4f1687f46d082c6d96ae/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:43.436794 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a7413-1879-49ae-aa69-fa850464ee5a admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:43.471185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2965531c-ea25-42ca-8cc2-5aacd1efe281 admin admin] The token cache is being invalidate because project f20b9baaa44d465194a8368fe6acbd72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:43.471438 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2965531c-ea25-42ca-8cc2-5aacd1efe281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:43.472132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2965531c-ea25-42ca-8cc2-5aacd1efe281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:43.474003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 940/1866] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:43 2022] DELETE /identity/v3/projects/f20b9baaa44d465194a8368fe6acbd72 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:43.477865 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965531c-ea25-42ca-8cc2-5aacd1efe281 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:43.479326 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2965531c-ea25-42ca-8cc2-5aacd1efe281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:43.551613 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a7413-1879-49ae-aa69-fa850464ee5a admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:43.564163 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a2450-c851-4b69-be9b-ff822aa41f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:43.564885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a2450-c851-4b69-be9b-ff822aa41f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:43.565180 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a2450-c851-4b69-be9b-ff822aa41f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:43.565798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a2450-c851-4b69-be9b-ff822aa41f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:43.566196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a2450-c851-4b69-be9b-ff822aa41f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:43.572915 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a2450-c851-4b69-be9b-ff822aa41f18 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:43.597381 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965531c-ea25-42ca-8cc2-5aacd1efe281 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:43.602590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47069e4-13c5-4255-a89d-39c0bc752a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:43.606408 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47069e4-13c5-4255-a89d-39c0bc752a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:43.606408 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47069e4-13c5-4255-a89d-39c0bc752a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:43.606408 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47069e4-13c5-4255-a89d-39c0bc752a71 admin admin] PATH_INFO: `/v3/users/11e03f13344c495881baf85fe15763ab` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:43.606408 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47069e4-13c5-4255-a89d-39c0bc752a71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e03f13344c495881baf85fe15763ab)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:43.607470 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47069e4-13c5-4255-a89d-39c0bc752a71 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:43.607874 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 927/1867] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:22:43.612621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a2450-c851-4b69-be9b-ff822aa41f18 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:43.615705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-af9a2450-c851-4b69-be9b-ff822aa41f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:43.635083 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a2450-c851-4b69-be9b-ff822aa41f18 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:43.641586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e55b78-1ba3-4e64-9a66-1eb4618a4a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:43.642503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e55b78-1ba3-4e64-9a66-1eb4618a4a79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:43.642795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e55b78-1ba3-4e64-9a66-1eb4618a4a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:43.643076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e55b78-1ba3-4e64-9a66-1eb4618a4a79 admin admin] PATH_INFO: `/v3/projects/cda77967154e49b6aeb6f90e64a3e31c/users/18a90c9c793a4f1687f46d082c6d96ae/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:43.645239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e55b78-1ba3-4e64-9a66-1eb4618a4a79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda77967154e49b6aeb6f90e64a3e31c, user_id=18a90c9c793a4f1687f46d082c6d96ae, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:43.648232 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e55b78-1ba3-4e64-9a66-1eb4618a4a79 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:43.654889 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:43.666264 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 928/1868] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:43 2022] PUT /identity/v3/projects/cda77967154e49b6aeb6f90e64a3e31c/users/18a90c9c793a4f1687f46d082c6d96ae/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:43.704837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db6695-fe3d-4b30-885a-e9ebcfaaab56 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:43.704837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db6695-fe3d-4b30-885a-e9ebcfaaab56 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:43.704837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db6695-fe3d-4b30-885a-e9ebcfaaab56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:43.792938 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a5db6695-fe3d-4b30-885a-e9ebcfaaab56 None None] MFA Rules not processed for user `18a90c9c793a4f1687f46d082c6d96ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:43.794269 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a5db6695-fe3d-4b30-885a-e9ebcfaaab56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:43.862334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f47069e4-13c5-4255-a89d-39c0bc752a71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e03f13344c495881baf85fe15763ab'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:43.884018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f47069e4-13c5-4255-a89d-39c0bc752a71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e03f13344c495881baf85fe15763ab'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:43.895294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 929/1869] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:43 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:43.952306 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db6695-fe3d-4b30-885a-e9ebcfaaab56 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:43.955289 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a5db6695-fe3d-4b30-885a-e9ebcfaaab56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:43.986066 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f47069e4-13c5-4255-a89d-39c0bc752a71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e03f13344c495881baf85fe15763ab'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:43.994422 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f47069e4-13c5-4255-a89d-39c0bc752a71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e03f13344c495881baf85fe15763ab'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:43.995228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f47069e4-13c5-4255-a89d-39c0bc752a71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e03f13344c495881baf85fe15763ab'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:44.003151 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 941/1870] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:43 2022] DELETE /identity/v3/users/11e03f13344c495881baf85fe15763ab => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:44.039119 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db6695-fe3d-4b30-885a-e9ebcfaaab56 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:44.111438 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56c272-d6fc-4930-9f9f-3c2af252a525 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:44.112668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56c272-d6fc-4930-9f9f-3c2af252a525 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:44.113796 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56c272-d6fc-4930-9f9f-3c2af252a525 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:44.114295 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56c272-d6fc-4930-9f9f-3c2af252a525 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:44.117158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ad56c272-d6fc-4930-9f9f-3c2af252a525 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:44.177077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:44.190932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad56c272-d6fc-4930-9f9f-3c2af252a525 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:44.192948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad56c272-d6fc-4930-9f9f-3c2af252a525 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:44.202549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 930/1871] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:22:44.363512 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:44.510211 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47069e4-13c5-4255-a89d-39c0bc752a71 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:44.512453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f47069e4-13c5-4255-a89d-39c0bc752a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:44.606804 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47069e4-13c5-4255-a89d-39c0bc752a71 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:44.611807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3e0fa-57f2-495e-8ed9-b6311eb1e1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:44.612700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3e0fa-57f2-495e-8ed9-b6311eb1e1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:44.613048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3e0fa-57f2-495e-8ed9-b6311eb1e1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:44.613291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3e0fa-57f2-495e-8ed9-b6311eb1e1e6 admin admin] PATH_INFO: `/v3/projects/a51add7be5884f53b41c7017fc2d5970` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:44.614550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3e0fa-57f2-495e-8ed9-b6311eb1e1e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a51add7be5884f53b41c7017fc2d5970)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:44.616611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3e0fa-57f2-495e-8ed9-b6311eb1e1e6 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:44.778907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0fd3e0fa-57f2-495e-8ed9-b6311eb1e1e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a51add7be5884f53b41c7017fc2d5970', 'request_id': 'req-0fd3e0fa-57f2-495e-8ed9-b6311eb1e1e6'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:44.786772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0fd3e0fa-57f2-495e-8ed9-b6311eb1e1e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a51add7be5884f53b41c7017fc2d5970', 'request_id': 'req-0fd3e0fa-57f2-495e-8ed9-b6311eb1e1e6'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:44.792540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0fd3e0fa-57f2-495e-8ed9-b6311eb1e1e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a51add7be5884f53b41c7017fc2d5970', 'request_id': 'req-0fd3e0fa-57f2-495e-8ed9-b6311eb1e1e6'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:44.854317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:44.922323 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0fd3e0fa-57f2-495e-8ed9-b6311eb1e1e6 admin admin] The token cache is being invalidate because project a51add7be5884f53b41c7017fc2d5970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:44.922935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0fd3e0fa-57f2-495e-8ed9-b6311eb1e1e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:44.923717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0fd3e0fa-57f2-495e-8ed9-b6311eb1e1e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:44.925337 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 942/1872] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:44 2022] DELETE /identity/v3/projects/a51add7be5884f53b41c7017fc2d5970 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:44.930258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc6ea4-a1c5-442a-8032-afad1d581608 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:44.930745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc6ea4-a1c5-442a-8032-afad1d581608 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:44.930831 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc6ea4-a1c5-442a-8032-afad1d581608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:44.949904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-65dc6ea4-a1c5-442a-8032-afad1d581608 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:44.951132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-65dc6ea4-a1c5-442a-8032-afad1d581608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:44.980869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:45.049241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 931/1873] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:45.058271 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d861d56-636b-4111-bc15-52e8d0265fc6 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:45.058857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d861d56-636b-4111-bc15-52e8d0265fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:45.059222 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d861d56-636b-4111-bc15-52e8d0265fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:45.089466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-6d861d56-636b-4111-bc15-52e8d0265fc6 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:45.089466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6d861d56-636b-4111-bc15-52e8d0265fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:45.098670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 943/1874] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:45.103012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc6ea4-a1c5-442a-8032-afad1d581608 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:45.117406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc6ea4-a1c5-442a-8032-afad1d581608 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:45.123739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb30175-f0a5-4108-ac11-9628168749fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:45.124580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb30175-f0a5-4108-ac11-9628168749fd admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:45.124870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb30175-f0a5-4108-ac11-9628168749fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:45.125183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb30175-f0a5-4108-ac11-9628168749fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:45.126482 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb30175-f0a5-4108-ac11-9628168749fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:45.126876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb30175-f0a5-4108-ac11-9628168749fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:45.128776 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb30175-f0a5-4108-ac11-9628168749fd admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:45.144058 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 932/1875] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:45.148330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d861d56-636b-4111-bc15-52e8d0265fc6 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:45.160154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d861d56-636b-4111-bc15-52e8d0265fc6 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:45.165860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2f3ea-9330-422e-b4de-916e828fe877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:45.166843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2f3ea-9330-422e-b4de-916e828fe877 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:45.167278 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2f3ea-9330-422e-b4de-916e828fe877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:45.167586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2f3ea-9330-422e-b4de-916e828fe877 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:45.169946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2f3ea-9330-422e-b4de-916e828fe877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:45.170386 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2f3ea-9330-422e-b4de-916e828fe877 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:45.276895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 944/1876] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:45 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:45.279778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb30175-f0a5-4108-ac11-9628168749fd admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:45.298685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb30175-f0a5-4108-ac11-9628168749fd admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:45.356576 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:45.370706 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c08855-2050-488a-a4be-8344f7b6195e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:45.371448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c08855-2050-488a-a4be-8344f7b6195e admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:45.371772 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c08855-2050-488a-a4be-8344f7b6195e admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:45.372107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c08855-2050-488a-a4be-8344f7b6195e admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:45.372850 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c08855-2050-488a-a4be-8344f7b6195e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:45.374925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c08855-2050-488a-a4be-8344f7b6195e admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:45.454948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 933/1877] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:45 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:45.459551 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2f3ea-9330-422e-b4de-916e828fe877 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:45.471820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2f3ea-9330-422e-b4de-916e828fe877 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:45.474634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:45.478788 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262b108-25e4-498a-a85c-4b8bd5067402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:45.479776 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262b108-25e4-498a-a85c-4b8bd5067402 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:45.479776 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262b108-25e4-498a-a85c-4b8bd5067402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:45.480146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262b108-25e4-498a-a85c-4b8bd5067402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:45.480817 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262b108-25e4-498a-a85c-4b8bd5067402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:45.482430 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262b108-25e4-498a-a85c-4b8bd5067402 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:45.493867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 945/1878] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:22:45.498232 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c08855-2050-488a-a4be-8344f7b6195e admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:45.509291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c08855-2050-488a-a4be-8344f7b6195e admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:45.514239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f0c65-8aff-4591-97c8-5185c3cb5024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:45.514987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f0c65-8aff-4591-97c8-5185c3cb5024 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:45.515346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f0c65-8aff-4591-97c8-5185c3cb5024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:45.515716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f0c65-8aff-4591-97c8-5185c3cb5024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:45.516531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f0c65-8aff-4591-97c8-5185c3cb5024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:45.518172 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f0c65-8aff-4591-97c8-5185c3cb5024 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:45.535280 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 934/1879] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:45.535455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262b108-25e4-498a-a85c-4b8bd5067402 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:45.544436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262b108-25e4-498a-a85c-4b8bd5067402 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:45.551233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db0dfb-cc63-40b7-a108-08f25e63e5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:45.552178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db0dfb-cc63-40b7-a108-08f25e63e5ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:45.552422 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db0dfb-cc63-40b7-a108-08f25e63e5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:45.552736 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db0dfb-cc63-40b7-a108-08f25e63e5ab admin admin] PATH_INFO: `/v3/projects/d69c473946e241f2a34235b6e405226e/users/ccdbaa8a89034237b8bd0e513be51d64/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:45.569663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db0dfb-cc63-40b7-a108-08f25e63e5ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69c473946e241f2a34235b6e405226e, user_id=ccdbaa8a89034237b8bd0e513be51d64, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:45.572353 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db0dfb-cc63-40b7-a108-08f25e63e5ab admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:45.668825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 946/1880] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:45 2022] PUT /identity/v3/projects/d69c473946e241f2a34235b6e405226e/users/ccdbaa8a89034237b8bd0e513be51d64/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:45.695706 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187bb7a-2017-4a2c-9a45-ef6ab58b1efc None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:45.696124 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187bb7a-2017-4a2c-9a45-ef6ab58b1efc None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:45.696449 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187bb7a-2017-4a2c-9a45-ef6ab58b1efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:45.733265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b187bb7a-2017-4a2c-9a45-ef6ab58b1efc None None] MFA Rules not processed for user `ccdbaa8a89034237b8bd0e513be51d64`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:45.734525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b187bb7a-2017-4a2c-9a45-ef6ab58b1efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:45.810653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 935/1881] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:45.815181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db0dfb-cc63-40b7-a108-08f25e63e5ab admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:45.917213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db0dfb-cc63-40b7-a108-08f25e63e5ab admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:45.934716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3f331-d393-4242-9e8f-bec8083dd727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:45.934935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3f331-d393-4242-9e8f-bec8083dd727 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:45.939172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3f331-d393-4242-9e8f-bec8083dd727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:45.939172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3f331-d393-4242-9e8f-bec8083dd727 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:45.939172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3f331-d393-4242-9e8f-bec8083dd727 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:45.942698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3f331-d393-4242-9e8f-bec8083dd727 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:46.115659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 947/1882] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:45 2022] POST /identity/v3/projects => generated 387 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:46.119939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187bb7a-2017-4a2c-9a45-ef6ab58b1efc None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:46.138400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187bb7a-2017-4a2c-9a45-ef6ab58b1efc None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:46.150040 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:46.226052 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:46.231738 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740e47c-1453-4c92-99d4-57fb6fbd288c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:46.232432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740e47c-1453-4c92-99d4-57fb6fbd288c admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:46.232739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740e47c-1453-4c92-99d4-57fb6fbd288c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:46.233029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740e47c-1453-4c92-99d4-57fb6fbd288c admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:46.233744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740e47c-1453-4c92-99d4-57fb6fbd288c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:46.237546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740e47c-1453-4c92-99d4-57fb6fbd288c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:46.324075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 936/1883] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:46 2022] POST /identity/v3/users => generated 300 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:22:46.329198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3f331-d393-4242-9e8f-bec8083dd727 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:46.341397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3f331-d393-4242-9e8f-bec8083dd727 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:46.348047 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdee8c-32b4-4329-aec9-6e9c6f168675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:46.348740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdee8c-32b4-4329-aec9-6e9c6f168675 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:46.349058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdee8c-32b4-4329-aec9-6e9c6f168675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:46.349337 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdee8c-32b4-4329-aec9-6e9c6f168675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:46.350895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbdee8c-32b4-4329-aec9-6e9c6f168675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:46.352006 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbdee8c-32b4-4329-aec9-6e9c6f168675 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:46.367219 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 948/1884] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:46.372699 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740e47c-1453-4c92-99d4-57fb6fbd288c admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:46.384329 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740e47c-1453-4c92-99d4-57fb6fbd288c admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:46.389440 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14e36c-d7fc-494a-b764-d6529499f047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:46.390493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14e36c-d7fc-494a-b764-d6529499f047 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:46.390631 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14e36c-d7fc-494a-b764-d6529499f047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:46.390901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b14e36c-d7fc-494a-b764-d6529499f047 admin admin] PATH_INFO: `/v3/projects/4ea7e7ec3f88469996d6d96758498a7d/users/e6c5bfefb88d4ccfb21517239d7bc1e9/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:46.411669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14e36c-d7fc-494a-b764-d6529499f047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea7e7ec3f88469996d6d96758498a7d, user_id=e6c5bfefb88d4ccfb21517239d7bc1e9, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:46.415240 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b14e36c-d7fc-494a-b764-d6529499f047 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:46.515777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 937/1885] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:46 2022] PUT /identity/v3/projects/4ea7e7ec3f88469996d6d96758498a7d/users/e6c5bfefb88d4ccfb21517239d7bc1e9/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:46.517026 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdee8c-32b4-4329-aec9-6e9c6f168675 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:46.528183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdee8c-32b4-4329-aec9-6e9c6f168675 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:46.538543 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:46.612766 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe35701-94e8-4e48-ade3-974fdb35c8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:46.613497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe35701-94e8-4e48-ade3-974fdb35c8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:46.613857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe35701-94e8-4e48-ade3-974fdb35c8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:46.614051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe35701-94e8-4e48-ade3-974fdb35c8f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:46.614800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe35701-94e8-4e48-ade3-974fdb35c8f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:46.616409 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe35701-94e8-4e48-ade3-974fdb35c8f8 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:46.638630 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 949/1886] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:22:46.644664 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14e36c-d7fc-494a-b764-d6529499f047 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:46.659241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b14e36c-d7fc-494a-b764-d6529499f047 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:46.667237 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2f3c6-d769-48b9-8fd9-75edb3382c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:46.668756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2f3c6-d769-48b9-8fd9-75edb3382c85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:46.669218 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2f3c6-d769-48b9-8fd9-75edb3382c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:46.669675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2f3c6-d769-48b9-8fd9-75edb3382c85 admin admin] PATH_INFO: `/v3/projects/4ea7e7ec3f88469996d6d96758498a7d/users/e6c5bfefb88d4ccfb21517239d7bc1e9/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:46.671971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2f3c6-d769-48b9-8fd9-75edb3382c85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ea7e7ec3f88469996d6d96758498a7d, user_id=e6c5bfefb88d4ccfb21517239d7bc1e9, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:46.678089 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2f3c6-d769-48b9-8fd9-75edb3382c85 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:46.703288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 938/1887] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:46 2022] PUT /identity/v3/projects/4ea7e7ec3f88469996d6d96758498a7d/users/e6c5bfefb88d4ccfb21517239d7bc1e9/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:46.714885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77055caa-10cd-4bd9-81d5-ffa6bc94c6d0 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:46.715175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77055caa-10cd-4bd9-81d5-ffa6bc94c6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:46.715458 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77055caa-10cd-4bd9-81d5-ffa6bc94c6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:46.747581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-77055caa-10cd-4bd9-81d5-ffa6bc94c6d0 None None] MFA Rules not processed for user `e6c5bfefb88d4ccfb21517239d7bc1e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:46.748871 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-77055caa-10cd-4bd9-81d5-ffa6bc94c6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:46.872013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 950/1888] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:46.885080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2f3c6-d769-48b9-8fd9-75edb3382c85 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:46.887442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4fa2f3c6-d769-48b9-8fd9-75edb3382c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:46.950061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2f3c6-d769-48b9-8fd9-75edb3382c85 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:46.954624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c284ccc-8e35-44b9-bc09-4666ad4e68d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:46.955325 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c284ccc-8e35-44b9-bc09-4666ad4e68d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:46.955588 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c284ccc-8e35-44b9-bc09-4666ad4e68d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:46.955867 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c284ccc-8e35-44b9-bc09-4666ad4e68d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:46.965973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c284ccc-8e35-44b9-bc09-4666ad4e68d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:46.967366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c284ccc-8e35-44b9-bc09-4666ad4e68d4 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:46.975598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:46.981234 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 939/1889] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:47.012930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77055caa-10cd-4bd9-81d5-ffa6bc94c6d0 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:47.089426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77055caa-10cd-4bd9-81d5-ffa6bc94c6d0 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:47.096704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cabadc-4a8f-4915-960a-5c44ec382c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:47.096704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cabadc-4a8f-4915-960a-5c44ec382c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:47.096704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cabadc-4a8f-4915-960a-5c44ec382c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:47.096704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cabadc-4a8f-4915-960a-5c44ec382c95 admin admin] PATH_INFO: `/v3/users/ccdbaa8a89034237b8bd0e513be51d64` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:47.098080 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cabadc-4a8f-4915-960a-5c44ec382c95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccdbaa8a89034237b8bd0e513be51d64)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:47.100239 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cabadc-4a8f-4915-960a-5c44ec382c95 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:47.240097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:47.334619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-44cabadc-4a8f-4915-960a-5c44ec382c95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccdbaa8a89034237b8bd0e513be51d64'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:47.351514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-44cabadc-4a8f-4915-960a-5c44ec382c95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccdbaa8a89034237b8bd0e513be51d64'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:47.423554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-44cabadc-4a8f-4915-960a-5c44ec382c95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccdbaa8a89034237b8bd0e513be51d64'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:47.432027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-44cabadc-4a8f-4915-960a-5c44ec382c95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccdbaa8a89034237b8bd0e513be51d64'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:47.432766 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-44cabadc-4a8f-4915-960a-5c44ec382c95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccdbaa8a89034237b8bd0e513be51d64'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:47.439707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 951/1890] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:47 2022] DELETE /identity/v3/users/ccdbaa8a89034237b8bd0e513be51d64 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:47.444813 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94b3a5-9fb0-4331-a46d-bba4593ffd74 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:47.445354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94b3a5-9fb0-4331-a46d-bba4593ffd74 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:47.445455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94b3a5-9fb0-4331-a46d-bba4593ffd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:47.464195 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-1f94b3a5-9fb0-4331-a46d-bba4593ffd74 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:47.467103 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1f94b3a5-9fb0-4331-a46d-bba4593ffd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:47.565274 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 940/1891] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:47.579983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cabadc-4a8f-4915-960a-5c44ec382c95 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:47.582581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-44cabadc-4a8f-4915-960a-5c44ec382c95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:47.657771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:47.688892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cabadc-4a8f-4915-960a-5c44ec382c95 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:47.709718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94b3a5-9fb0-4331-a46d-bba4593ffd74 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:47.709718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1f94b3a5-9fb0-4331-a46d-bba4593ffd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:47.710331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e57567-fa44-4e05-bda0-cf7f1c311156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:47.710331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e57567-fa44-4e05-bda0-cf7f1c311156 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:47.710331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e57567-fa44-4e05-bda0-cf7f1c311156 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:47.710331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e57567-fa44-4e05-bda0-cf7f1c311156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:47.719869 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e57567-fa44-4e05-bda0-cf7f1c311156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:47.721227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e57567-fa44-4e05-bda0-cf7f1c311156 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:47.734058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 952/1892] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:47.791550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:47.794540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94b3a5-9fb0-4331-a46d-bba4593ffd74 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:47.799222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf6230-9ab2-4824-9e1d-740b6a885014 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:47.800670 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf6230-9ab2-4824-9e1d-740b6a885014 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:47.801050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf6230-9ab2-4824-9e1d-740b6a885014 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:47.801438 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf6230-9ab2-4824-9e1d-740b6a885014 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:47.803797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-cbbf6230-9ab2-4824-9e1d-740b6a885014 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:47.855238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:47.894686 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e57567-fa44-4e05-bda0-cf7f1c311156 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:47.898841 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e6e57567-fa44-4e05-bda0-cf7f1c311156 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:47.928565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e57567-fa44-4e05-bda0-cf7f1c311156 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:47.938316 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd29b4a-4c96-4f4b-aace-f1fc66561611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:47.940719 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd29b4a-4c96-4f4b-aace-f1fc66561611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:47.941270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd29b4a-4c96-4f4b-aace-f1fc66561611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:47.942567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd29b4a-4c96-4f4b-aace-f1fc66561611 admin admin] PATH_INFO: `/v3/users/0c08efdef298498592a77dfe5e85f717` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:47.943108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf6230-9ab2-4824-9e1d-740b6a885014 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:47.944784 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf6230-9ab2-4824-9e1d-740b6a885014 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:47.945742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd29b4a-4c96-4f4b-aace-f1fc66561611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c08efdef298498592a77dfe5e85f717)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:47.950949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd29b4a-4c96-4f4b-aace-f1fc66561611 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:47.956200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 941/1893] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:47.992317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:48.151627 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:48.216712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2dd29b4a-4c96-4f4b-aace-f1fc66561611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c08efdef298498592a77dfe5e85f717'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:48.232563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2dd29b4a-4c96-4f4b-aace-f1fc66561611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c08efdef298498592a77dfe5e85f717'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:48.305643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2dd29b4a-4c96-4f4b-aace-f1fc66561611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c08efdef298498592a77dfe5e85f717'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:48.313812 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2dd29b4a-4c96-4f4b-aace-f1fc66561611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c08efdef298498592a77dfe5e85f717'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:48.314498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2dd29b4a-4c96-4f4b-aace-f1fc66561611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c08efdef298498592a77dfe5e85f717'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:48.326516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 953/1894] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:47 2022] DELETE /identity/v3/users/0c08efdef298498592a77dfe5e85f717 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:48.326694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9ed0f-36e6-47c3-b602-590cbcd46ab1 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:48.326694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9ed0f-36e6-47c3-b602-590cbcd46ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:48.326694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9ed0f-36e6-47c3-b602-590cbcd46ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:48.345584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-bfd9ed0f-36e6-47c3-b602-590cbcd46ab1 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:48.346674 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-bfd9ed0f-36e6-47c3-b602-590cbcd46ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:48.390152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd29b4a-4c96-4f4b-aace-f1fc66561611 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:48.394031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2dd29b4a-4c96-4f4b-aace-f1fc66561611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:48.413760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 942/1895] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:48.443954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9ed0f-36e6-47c3-b602-590cbcd46ab1 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:48.445516 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-bfd9ed0f-36e6-47c3-b602-590cbcd46ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:48.521335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9ed0f-36e6-47c3-b602-590cbcd46ab1 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:48.525496 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df455a-473e-45a2-bfbf-4a73040d6ae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:48.526437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df455a-473e-45a2-bfbf-4a73040d6ae6 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:48.526796 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df455a-473e-45a2-bfbf-4a73040d6ae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:48.527279 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df455a-473e-45a2-bfbf-4a73040d6ae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:48.539946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df455a-473e-45a2-bfbf-4a73040d6ae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:48.541758 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df455a-473e-45a2-bfbf-4a73040d6ae6 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:48.554640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 943/1896] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:22:48.593977 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:48.628830 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd29b4a-4c96-4f4b-aace-f1fc66561611 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:48.633681 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:48.634628 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:48.634916 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:48.635186 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851 admin admin] PATH_INFO: `/v3/projects/d69c473946e241f2a34235b6e405226e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:48.636389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d69c473946e241f2a34235b6e405226e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:48.638348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:48.667070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:48.777777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd69c473946e241f2a34235b6e405226e', 'request_id': 'req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:48.784766 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd69c473946e241f2a34235b6e405226e', 'request_id': 'req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:48.791275 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd69c473946e241f2a34235b6e405226e', 'request_id': 'req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:48.871217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851 admin admin] The token cache is being invalidate because project d69c473946e241f2a34235b6e405226e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:48.871217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:48.872516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:48.875334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 954/1897] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:48 2022] DELETE /identity/v3/projects/d69c473946e241f2a34235b6e405226e => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:48.880623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df455a-473e-45a2-bfbf-4a73040d6ae6 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:48.893364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df455a-473e-45a2-bfbf-4a73040d6ae6 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:48.952485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:48.978649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5855052-6374-495e-ae0c-8165f93b5657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:48.979556 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5855052-6374-495e-ae0c-8165f93b5657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:48.979842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5855052-6374-495e-ae0c-8165f93b5657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:48.980168 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5855052-6374-495e-ae0c-8165f93b5657 admin admin] PATH_INFO: `/v3/users/e6c5bfefb88d4ccfb21517239d7bc1e9` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:48.983011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5855052-6374-495e-ae0c-8165f93b5657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6c5bfefb88d4ccfb21517239d7bc1e9)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:48.991148 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5855052-6374-495e-ae0c-8165f93b5657 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:49.076909 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:49.079292 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:49.116119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9e0e6-0762-4175-9f1d-f8a80c0d8851 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:49.122538 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e9953-8f66-4841-bdcd-a9b00cb5c61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:49.123324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e9953-8f66-4841-bdcd-a9b00cb5c61a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:49.123615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e9953-8f66-4841-bdcd-a9b00cb5c61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:49.123893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e9953-8f66-4841-bdcd-a9b00cb5c61a admin admin] PATH_INFO: `/v3/projects/7cae8db77a4d4647b349d0bd4dd056ba` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:49.125246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e9953-8f66-4841-bdcd-a9b00cb5c61a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cae8db77a4d4647b349d0bd4dd056ba)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:49.127936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e9953-8f66-4841-bdcd-a9b00cb5c61a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:49.227120 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b5855052-6374-495e-ae0c-8165f93b5657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6c5bfefb88d4ccfb21517239d7bc1e9'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:49.303717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d64e9953-8f66-4841-bdcd-a9b00cb5c61a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cae8db77a4d4647b349d0bd4dd056ba', 'request_id': 'req-d64e9953-8f66-4841-bdcd-a9b00cb5c61a'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:49.316224 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b5855052-6374-495e-ae0c-8165f93b5657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6c5bfefb88d4ccfb21517239d7bc1e9'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:49.318896 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b5855052-6374-495e-ae0c-8165f93b5657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6c5bfefb88d4ccfb21517239d7bc1e9'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:49.319690 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b5855052-6374-495e-ae0c-8165f93b5657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6c5bfefb88d4ccfb21517239d7bc1e9'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:49.321475 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d64e9953-8f66-4841-bdcd-a9b00cb5c61a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cae8db77a4d4647b349d0bd4dd056ba', 'request_id': 'req-d64e9953-8f66-4841-bdcd-a9b00cb5c61a'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:49.331211 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d64e9953-8f66-4841-bdcd-a9b00cb5c61a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cae8db77a4d4647b349d0bd4dd056ba', 'request_id': 'req-d64e9953-8f66-4841-bdcd-a9b00cb5c61a'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:49.337435 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b5855052-6374-495e-ae0c-8165f93b5657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6c5bfefb88d4ccfb21517239d7bc1e9'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:49.355897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 944/1898] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:48 2022] DELETE /identity/v3/users/e6c5bfefb88d4ccfb21517239d7bc1e9 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:49.450439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d64e9953-8f66-4841-bdcd-a9b00cb5c61a admin admin] The token cache is being invalidate because project 7cae8db77a4d4647b349d0bd4dd056ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:49.451323 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d64e9953-8f66-4841-bdcd-a9b00cb5c61a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:49.452198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d64e9953-8f66-4841-bdcd-a9b00cb5c61a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:49.457362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 955/1899] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:49 2022] DELETE /identity/v3/projects/7cae8db77a4d4647b349d0bd4dd056ba => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:49.463146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160da077-733e-4109-b4ad-15cafa7ad1e1 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:49.464045 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160da077-733e-4109-b4ad-15cafa7ad1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:49.464832 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160da077-733e-4109-b4ad-15cafa7ad1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:49.491564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-160da077-733e-4109-b4ad-15cafa7ad1e1 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:49.491981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-160da077-733e-4109-b4ad-15cafa7ad1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:49.531164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:49.560126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 945/1900] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:49.568864 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c095c-5b8b-408c-8c73-9aa7f0ff574a None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:49.569192 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c095c-5b8b-408c-8c73-9aa7f0ff574a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:49.569458 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c095c-5b8b-408c-8c73-9aa7f0ff574a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:49.588728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-096c095c-5b8b-408c-8c73-9aa7f0ff574a None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:49.589982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-096c095c-5b8b-408c-8c73-9aa7f0ff574a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:49.599394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 956/1901] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:49.606973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160da077-733e-4109-b4ad-15cafa7ad1e1 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:49.615745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160da077-733e-4109-b4ad-15cafa7ad1e1 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:49.621550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2229452-0c55-42a8-9816-75c4fd90c4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:49.622417 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2229452-0c55-42a8-9816-75c4fd90c4c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:49.622682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2229452-0c55-42a8-9816-75c4fd90c4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:49.623041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2229452-0c55-42a8-9816-75c4fd90c4c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:49.624034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2229452-0c55-42a8-9816-75c4fd90c4c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:49.624432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2229452-0c55-42a8-9816-75c4fd90c4c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:49.626812 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2229452-0c55-42a8-9816-75c4fd90c4c9 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:49.643342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 946/1902] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:49.645416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c095c-5b8b-408c-8c73-9aa7f0ff574a None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:49.655789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c095c-5b8b-408c-8c73-9aa7f0ff574a None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:49.661052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c031e2-5d92-4a99-932b-bbe84549aed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:49.662383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c031e2-5d92-4a99-932b-bbe84549aed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:49.662913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c031e2-5d92-4a99-932b-bbe84549aed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:49.663260 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c031e2-5d92-4a99-932b-bbe84549aed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:49.664028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c031e2-5d92-4a99-932b-bbe84549aed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:49.666117 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c031e2-5d92-4a99-932b-bbe84549aed4 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:49.713570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2229452-0c55-42a8-9816-75c4fd90c4c9 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:49.715548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f2229452-0c55-42a8-9816-75c4fd90c4c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:49.754610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 957/1903] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:49 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:49.759161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c031e2-5d92-4a99-932b-bbe84549aed4 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:49.785296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2229452-0c55-42a8-9816-75c4fd90c4c9 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:49.845389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c031e2-5d92-4a99-932b-bbe84549aed4 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:49.849933 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcc2d8-b205-4549-8975-2eea67362f48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:49.850412 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615d748-730e-4044-87a7-99aa9cf660a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:49.850677 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcc2d8-b205-4549-8975-2eea67362f48 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:49.850962 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcc2d8-b205-4549-8975-2eea67362f48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:49.851092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615d748-730e-4044-87a7-99aa9cf660a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:49.851232 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcc2d8-b205-4549-8975-2eea67362f48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:49.851355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615d748-730e-4044-87a7-99aa9cf660a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:49.851625 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615d748-730e-4044-87a7-99aa9cf660a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:49.852369 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615d748-730e-4044-87a7-99aa9cf660a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:49.854166 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c6fcc2d8-b205-4549-8975-2eea67362f48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:49.854535 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615d748-730e-4044-87a7-99aa9cf660a5 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:49.865309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=81,keystone:DELETE=13,keystone:INSERT=6 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:49.925974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcc2d8-b205-4549-8975-2eea67362f48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:49.927560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcc2d8-b205-4549-8975-2eea67362f48 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:49.939233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 947/1904] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:49 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:22:49.940072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcc2d8-b205-4549-8975-2eea67362f48 service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:49.941724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c6fcc2d8-b205-4549-8975-2eea67362f48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:49.955704 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcc2d8-b205-4549-8975-2eea67362f48 service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:49.960811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a052cc-8dd5-456f-94c0-dd82663e5bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:49.961669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a052cc-8dd5-456f-94c0-dd82663e5bbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:49.961998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a052cc-8dd5-456f-94c0-dd82663e5bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:49.962277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a052cc-8dd5-456f-94c0-dd82663e5bbc admin admin] PATH_INFO: `/v3/projects/4ea7e7ec3f88469996d6d96758498a7d` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:49.963536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a052cc-8dd5-456f-94c0-dd82663e5bbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ea7e7ec3f88469996d6d96758498a7d)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:49.965462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a052cc-8dd5-456f-94c0-dd82663e5bbc admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:49.997270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 958/1905] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:49 2022] POST /identity/v3/users => generated 287 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:50.001563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615d748-730e-4044-87a7-99aa9cf660a5 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:50.015782 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615d748-730e-4044-87a7-99aa9cf660a5 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:50.021224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804855a-d6c9-4248-a8ad-c4cbbcf837c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:50.029527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804855a-d6c9-4248-a8ad-c4cbbcf837c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:50.029527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804855a-d6c9-4248-a8ad-c4cbbcf837c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:50.029527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804855a-d6c9-4248-a8ad-c4cbbcf837c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:50.029527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804855a-d6c9-4248-a8ad-c4cbbcf837c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:50.029527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804855a-d6c9-4248-a8ad-c4cbbcf837c6 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:50.036359 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 959/1906] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:22:50.040469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804855a-d6c9-4248-a8ad-c4cbbcf837c6 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:50.052128 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804855a-d6c9-4248-a8ad-c4cbbcf837c6 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:50.056840 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d0c96-6edb-4330-baec-22cddcf76fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:50.057527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d0c96-6edb-4330-baec-22cddcf76fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:50.058173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d0c96-6edb-4330-baec-22cddcf76fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:50.058467 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d0c96-6edb-4330-baec-22cddcf76fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:50.059194 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d0c96-6edb-4330-baec-22cddcf76fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:50.060824 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d0c96-6edb-4330-baec-22cddcf76fa2 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:50.073094 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 960/1907] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:22:50.077100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d0c96-6edb-4330-baec-22cddcf76fa2 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:50.092127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d0c96-6edb-4330-baec-22cddcf76fa2 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:50.098278 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1f51b-9aae-4fde-8c56-a04f4cc25223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:50.099164 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1f51b-9aae-4fde-8c56-a04f4cc25223 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:50.099443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1f51b-9aae-4fde-8c56-a04f4cc25223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:50.099716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1f51b-9aae-4fde-8c56-a04f4cc25223 admin admin] PATH_INFO: `/v3/projects/358a43481125457cb095446b7daa9bef/users/575156e97837437596b3f8711ba12484/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:50.118539 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1f51b-9aae-4fde-8c56-a04f4cc25223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=358a43481125457cb095446b7daa9bef, user_id=575156e97837437596b3f8711ba12484, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:50.121472 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1f51b-9aae-4fde-8c56-a04f4cc25223 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:50.127101 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-54a052cc-8dd5-456f-94c0-dd82663e5bbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ea7e7ec3f88469996d6d96758498a7d', 'request_id': 'req-54a052cc-8dd5-456f-94c0-dd82663e5bbc'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:50.133409 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-54a052cc-8dd5-456f-94c0-dd82663e5bbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ea7e7ec3f88469996d6d96758498a7d', 'request_id': 'req-54a052cc-8dd5-456f-94c0-dd82663e5bbc'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:50.214411 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-54a052cc-8dd5-456f-94c0-dd82663e5bbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ea7e7ec3f88469996d6d96758498a7d', 'request_id': 'req-54a052cc-8dd5-456f-94c0-dd82663e5bbc'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:50.238985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 961/1908] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:50 2022] PUT /identity/v3/projects/358a43481125457cb095446b7daa9bef/users/575156e97837437596b3f8711ba12484/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:50.240395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-54a052cc-8dd5-456f-94c0-dd82663e5bbc admin admin] The token cache is being invalidate because project 4ea7e7ec3f88469996d6d96758498a7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:50.240816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-54a052cc-8dd5-456f-94c0-dd82663e5bbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:50.241923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-54a052cc-8dd5-456f-94c0-dd82663e5bbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:50.242422 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7056a-f09f-4f0d-bccd-7db2bd42fc91 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:50.242864 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7056a-f09f-4f0d-bccd-7db2bd42fc91 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:50.243203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7056a-f09f-4f0d-bccd-7db2bd42fc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:50.256628 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 948/1909] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:49 2022] DELETE /identity/v3/projects/4ea7e7ec3f88469996d6d96758498a7d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:50.264331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f293ca-8ed3-43ae-8d82-65f3190aa7c7 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:50.264331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f293ca-8ed3-43ae-8d82-65f3190aa7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:50.264693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f293ca-8ed3-43ae-8d82-65f3190aa7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:50.277679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-bcc7056a-f09f-4f0d-bccd-7db2bd42fc91 None None] MFA Rules not processed for user `575156e97837437596b3f8711ba12484`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:50.278895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-bcc7056a-f09f-4f0d-bccd-7db2bd42fc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:50.308653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-18f293ca-8ed3-43ae-8d82-65f3190aa7c7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:50.309034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-18f293ca-8ed3-43ae-8d82-65f3190aa7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:50.387971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 962/1910] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:50.395977 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7056a-f09f-4f0d-bccd-7db2bd42fc91 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:50.398719 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-bcc7056a-f09f-4f0d-bccd-7db2bd42fc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:50.417443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 949/1911] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:50.427291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d1cd2-a26e-4658-aadb-bb618f5c3b6a None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:50.427584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d1cd2-a26e-4658-aadb-bb618f5c3b6a None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:50.427870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d1cd2-a26e-4658-aadb-bb618f5c3b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:50.451962 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-412d1cd2-a26e-4658-aadb-bb618f5c3b6a None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:50.453282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-412d1cd2-a26e-4658-aadb-bb618f5c3b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:50.466004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 950/1912] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:50.475703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7056a-f09f-4f0d-bccd-7db2bd42fc91 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:50.476761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d1cd2-a26e-4658-aadb-bb618f5c3b6a None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:50.479888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bab51-89c7-41a5-a401-d2022c1f73ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:50.480646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bab51-89c7-41a5-a401-d2022c1f73ed service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:50.480869 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bab51-89c7-41a5-a401-d2022c1f73ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:50.481154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bab51-89c7-41a5-a401-d2022c1f73ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:50.492543 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bab51-89c7-41a5-a401-d2022c1f73ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:50.493936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bab51-89c7-41a5-a401-d2022c1f73ed service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:50.503460 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d1cd2-a26e-4658-aadb-bb618f5c3b6a None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:50.508549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 963/1913] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:22:50.513319 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186326f7-276f-417d-832d-b09c05f3f0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:50.513960 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186326f7-276f-417d-832d-b09c05f3f0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:50.514251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186326f7-276f-417d-832d-b09c05f3f0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:50.514526 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186326f7-276f-417d-832d-b09c05f3f0a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:50.515279 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186326f7-276f-417d-832d-b09c05f3f0a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:50.515584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186326f7-276f-417d-832d-b09c05f3f0a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:50.518211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186326f7-276f-417d-832d-b09c05f3f0a0 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:50.534757 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 951/1914] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:22:50.539207 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bab51-89c7-41a5-a401-d2022c1f73ed service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:50.552094 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bab51-89c7-41a5-a401-d2022c1f73ed service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:50.553398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:50.557293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18283ddb-ad39-4a0c-bcf0-0a6c0869afff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:50.558004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18283ddb-ad39-4a0c-bcf0-0a6c0869afff admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:50.558288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18283ddb-ad39-4a0c-bcf0-0a6c0869afff admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:50.558562 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18283ddb-ad39-4a0c-bcf0-0a6c0869afff admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:50.559261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18283ddb-ad39-4a0c-bcf0-0a6c0869afff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:50.561592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18283ddb-ad39-4a0c-bcf0-0a6c0869afff admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:50.670757 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186326f7-276f-417d-832d-b09c05f3f0a0 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:50.678894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 964/1915] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:50 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:22:50.683810 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186326f7-276f-417d-832d-b09c05f3f0a0 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:50.767882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9d98b-8433-4995-9e05-a6546565e087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:50.769232 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9d98b-8433-4995-9e05-a6546565e087 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:50.769537 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9d98b-8433-4995-9e05-a6546565e087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:50.769817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9d98b-8433-4995-9e05-a6546565e087 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:50.770631 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9d98b-8433-4995-9e05-a6546565e087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:50.773721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9d98b-8433-4995-9e05-a6546565e087 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:50.890739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 952/1916] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:50 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:22:50.892471 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18283ddb-ad39-4a0c-bcf0-0a6c0869afff admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:50.906010 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18283ddb-ad39-4a0c-bcf0-0a6c0869afff admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:50.912437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b5a72-742f-4da8-8dfe-26cfaa8cf6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:50.913179 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b5a72-742f-4da8-8dfe-26cfaa8cf6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:50.913469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b5a72-742f-4da8-8dfe-26cfaa8cf6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:50.913749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b5a72-742f-4da8-8dfe-26cfaa8cf6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:50.914504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b5a72-742f-4da8-8dfe-26cfaa8cf6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:50.916178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b5a72-742f-4da8-8dfe-26cfaa8cf6a3 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:50.933016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 965/1917] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:50.937434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9d98b-8433-4995-9e05-a6546565e087 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:50.950357 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9d98b-8433-4995-9e05-a6546565e087 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:50.956991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598878f1-9406-494b-995c-6837fbcdb1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:50.957559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598878f1-9406-494b-995c-6837fbcdb1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:50.957829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598878f1-9406-494b-995c-6837fbcdb1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:50.960621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598878f1-9406-494b-995c-6837fbcdb1b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:50.960621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598878f1-9406-494b-995c-6837fbcdb1b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:50.960799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598878f1-9406-494b-995c-6837fbcdb1b5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:50.974029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 953/1918] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:50.978373 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b5a72-742f-4da8-8dfe-26cfaa8cf6a3 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:50.990577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b5a72-742f-4da8-8dfe-26cfaa8cf6a3 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:50.995259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb09834-08c8-4eb3-bdec-33ed88318ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:50.996161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb09834-08c8-4eb3-bdec-33ed88318ce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:50.996438 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb09834-08c8-4eb3-bdec-33ed88318ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:50.996738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb09834-08c8-4eb3-bdec-33ed88318ce2 admin admin] PATH_INFO: `/v3/projects/9b618d3911504c12a2b409f4cc732d91/users/7da5019b21ca4769a013a40975dcc3da/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:51.013427 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb09834-08c8-4eb3-bdec-33ed88318ce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b618d3911504c12a2b409f4cc732d91, user_id=7da5019b21ca4769a013a40975dcc3da, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:51.016187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb09834-08c8-4eb3-bdec-33ed88318ce2 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:51.060107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:51.118573 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 966/1919] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:50 2022] PUT /identity/v3/projects/9b618d3911504c12a2b409f4cc732d91/users/7da5019b21ca4769a013a40975dcc3da/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:51.129872 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103cfa9-a6b1-4466-b058-45a523f76e0d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:51.130163 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103cfa9-a6b1-4466-b058-45a523f76e0d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:51.130478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103cfa9-a6b1-4466-b058-45a523f76e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:51.160830 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-d103cfa9-a6b1-4466-b058-45a523f76e0d None None] MFA Rules not processed for user `7da5019b21ca4769a013a40975dcc3da`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:51.162849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d103cfa9-a6b1-4466-b058-45a523f76e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:51.210148 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:51.235781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 954/1920] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:51.248833 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb09834-08c8-4eb3-bdec-33ed88318ce2 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:51.250650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-cfb09834-08c8-4eb3-bdec-33ed88318ce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:51.316650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb09834-08c8-4eb3-bdec-33ed88318ce2 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:51.320815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5cb20-ac4a-4668-8dfb-b514966c2793 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:51.321645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5cb20-ac4a-4668-8dfb-b514966c2793 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:51.321804 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5cb20-ac4a-4668-8dfb-b514966c2793 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:51.325356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5cb20-ac4a-4668-8dfb-b514966c2793 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:51.339937 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5cb20-ac4a-4668-8dfb-b514966c2793 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:51.340795 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5cb20-ac4a-4668-8dfb-b514966c2793 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:51.351915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 967/1921] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:51.380176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103cfa9-a6b1-4466-b058-45a523f76e0d None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:51.491474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103cfa9-a6b1-4466-b058-45a523f76e0d None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:51.496719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c1a29-4c0e-4975-8842-164d3dce3174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:51.497688 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c1a29-4c0e-4975-8842-164d3dce3174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:51.497975 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c1a29-4c0e-4975-8842-164d3dce3174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:51.498257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c1a29-4c0e-4975-8842-164d3dce3174 admin admin] PATH_INFO: `/v3/users/7da5019b21ca4769a013a40975dcc3da` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:51.499733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c1a29-4c0e-4975-8842-164d3dce3174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7da5019b21ca4769a013a40975dcc3da)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:51.502437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c1a29-4c0e-4975-8842-164d3dce3174 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:51.791101 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-2a2c1a29-4c0e-4975-8842-164d3dce3174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7da5019b21ca4769a013a40975dcc3da'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:51.878684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-2a2c1a29-4c0e-4975-8842-164d3dce3174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7da5019b21ca4769a013a40975dcc3da'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:51.879355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-2a2c1a29-4c0e-4975-8842-164d3dce3174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7da5019b21ca4769a013a40975dcc3da'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:51.879985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-2a2c1a29-4c0e-4975-8842-164d3dce3174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7da5019b21ca4769a013a40975dcc3da'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:51.895078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-2a2c1a29-4c0e-4975-8842-164d3dce3174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7da5019b21ca4769a013a40975dcc3da'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:51.908537 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 955/1922] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:51 2022] DELETE /identity/v3/users/7da5019b21ca4769a013a40975dcc3da => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:51.913604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2885b9-83c8-4b39-8a62-347f618d82b7 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:51.914114 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2885b9-83c8-4b39-8a62-347f618d82b7 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:51.914236 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2885b9-83c8-4b39-8a62-347f618d82b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:51.934177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-ae2885b9-83c8-4b39-8a62-347f618d82b7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:51.935565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ae2885b9-83c8-4b39-8a62-347f618d82b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:52.019146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 968/1923] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:52.029268 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c1a29-4c0e-4975-8842-164d3dce3174 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:52.030913 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-2a2c1a29-4c0e-4975-8842-164d3dce3174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:52.087628 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:52.123247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c1a29-4c0e-4975-8842-164d3dce3174 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:52.128747 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b67bb7-8fa4-4e01-9e6b-c7694ddd3b4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:52.129781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b67bb7-8fa4-4e01-9e6b-c7694ddd3b4b service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:52.130235 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b67bb7-8fa4-4e01-9e6b-c7694ddd3b4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:52.130646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b67bb7-8fa4-4e01-9e6b-c7694ddd3b4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:52.141456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b67bb7-8fa4-4e01-9e6b-c7694ddd3b4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:52.143111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b67bb7-8fa4-4e01-9e6b-c7694ddd3b4b service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:52.155093 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 956/1924] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:52.599976 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2885b9-83c8-4b39-8a62-347f618d82b7 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:52.601436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ae2885b9-83c8-4b39-8a62-347f618d82b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:52.615537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2885b9-83c8-4b39-8a62-347f618d82b7 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:52.621058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69ce07-e68a-46d0-85e2-d73cc92cdc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:52.622003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69ce07-e68a-46d0-85e2-d73cc92cdc3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:52.622274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69ce07-e68a-46d0-85e2-d73cc92cdc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:52.622640 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69ce07-e68a-46d0-85e2-d73cc92cdc3e admin admin] PATH_INFO: `/v3/projects/9b618d3911504c12a2b409f4cc732d91` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:52.624068 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69ce07-e68a-46d0-85e2-d73cc92cdc3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b618d3911504c12a2b409f4cc732d91)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:52.626320 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69ce07-e68a-46d0-85e2-d73cc92cdc3e admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:52.931331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-dd69ce07-e68a-46d0-85e2-d73cc92cdc3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b618d3911504c12a2b409f4cc732d91', 'request_id': 'req-dd69ce07-e68a-46d0-85e2-d73cc92cdc3e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:52.938131 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-dd69ce07-e68a-46d0-85e2-d73cc92cdc3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b618d3911504c12a2b409f4cc732d91', 'request_id': 'req-dd69ce07-e68a-46d0-85e2-d73cc92cdc3e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:52.962120 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-dd69ce07-e68a-46d0-85e2-d73cc92cdc3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b618d3911504c12a2b409f4cc732d91', 'request_id': 'req-dd69ce07-e68a-46d0-85e2-d73cc92cdc3e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:53.192636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-dd69ce07-e68a-46d0-85e2-d73cc92cdc3e admin admin] The token cache is being invalidate because project 9b618d3911504c12a2b409f4cc732d91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:53.192636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-dd69ce07-e68a-46d0-85e2-d73cc92cdc3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:53.192636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-dd69ce07-e68a-46d0-85e2-d73cc92cdc3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:53.192636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 969/1925] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:52 2022] DELETE /identity/v3/projects/9b618d3911504c12a2b409f4cc732d91 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:53.193476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273ecd9-63b1-4659-b45b-f07e1a1f5463 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:53.193476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273ecd9-63b1-4659-b45b-f07e1a1f5463 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:53.193476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273ecd9-63b1-4659-b45b-f07e1a1f5463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:53.216342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-f273ecd9-63b1-4659-b45b-f07e1a1f5463 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:53.218767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f273ecd9-63b1-4659-b45b-f07e1a1f5463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:53.305386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 957/1926] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:53.307139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ae255-dc40-42d3-a83e-ac00478e764d None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:53.307516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ae255-dc40-42d3-a83e-ac00478e764d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:53.307821 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ae255-dc40-42d3-a83e-ac00478e764d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:53.332037 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-8f0ae255-dc40-42d3-a83e-ac00478e764d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:53.334418 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-8f0ae255-dc40-42d3-a83e-ac00478e764d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:53.353967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 970/1927] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:53.360636 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273ecd9-63b1-4659-b45b-f07e1a1f5463 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:53.375247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273ecd9-63b1-4659-b45b-f07e1a1f5463 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:53.381459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8978fe9-711d-4197-bc15-b0b43bf5f4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:53.382137 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8978fe9-711d-4197-bc15-b0b43bf5f4b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:53.382421 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8978fe9-711d-4197-bc15-b0b43bf5f4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:53.382698 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8978fe9-711d-4197-bc15-b0b43bf5f4b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:53.383648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8978fe9-711d-4197-bc15-b0b43bf5f4b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:53.383771 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8978fe9-711d-4197-bc15-b0b43bf5f4b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:53.385420 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8978fe9-711d-4197-bc15-b0b43bf5f4b1 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:53.405780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 958/1928] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:22:53.406161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ae255-dc40-42d3-a83e-ac00478e764d None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:53.412510 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:53.431399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ae255-dc40-42d3-a83e-ac00478e764d None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:53.440011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1863b-55c7-42e6-8465-f4622f95c326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:53.441556 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1863b-55c7-42e6-8465-f4622f95c326 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:53.441914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1863b-55c7-42e6-8465-f4622f95c326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:53.442178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1863b-55c7-42e6-8465-f4622f95c326 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:53.447804 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1863b-55c7-42e6-8465-f4622f95c326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:53.447804 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1863b-55c7-42e6-8465-f4622f95c326 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:53.560237 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 971/1929] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:53 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:22:53.564785 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8978fe9-711d-4197-bc15-b0b43bf5f4b1 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:53.594025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8978fe9-711d-4197-bc15-b0b43bf5f4b1 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:53.734008 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3908bd5-eb45-46c1-ae45-bd206d334813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:53.735055 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3908bd5-eb45-46c1-ae45-bd206d334813 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:53.735346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3908bd5-eb45-46c1-ae45-bd206d334813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:53.735692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3908bd5-eb45-46c1-ae45-bd206d334813 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:53.736410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3908bd5-eb45-46c1-ae45-bd206d334813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:53.738538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3908bd5-eb45-46c1-ae45-bd206d334813 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:53.853175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 959/1930] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:53 2022] POST /identity/v3/users => generated 287 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:53.857149 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1863b-55c7-42e6-8465-f4622f95c326 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:53.885450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1863b-55c7-42e6-8465-f4622f95c326 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:53.901697 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fa6ff-6bc8-4396-952d-38f3feb3688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:53.902425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fa6ff-6bc8-4396-952d-38f3feb3688e admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:53.902717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fa6ff-6bc8-4396-952d-38f3feb3688e admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:53.902999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fa6ff-6bc8-4396-952d-38f3feb3688e admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:53.903763 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fa6ff-6bc8-4396-952d-38f3feb3688e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:53.905723 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fa6ff-6bc8-4396-952d-38f3feb3688e admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:53.926998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 972/1931] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:53.932016 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3908bd5-eb45-46c1-ae45-bd206d334813 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:53.947472 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3908bd5-eb45-46c1-ae45-bd206d334813 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:53.957531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f5c8a-c299-4d67-9d65-7cc34a0378b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:53.959792 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f5c8a-c299-4d67-9d65-7cc34a0378b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:53.960300 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f5c8a-c299-4d67-9d65-7cc34a0378b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:53.961069 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f5c8a-c299-4d67-9d65-7cc34a0378b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:53.962485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f5c8a-c299-4d67-9d65-7cc34a0378b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:53.965969 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f5c8a-c299-4d67-9d65-7cc34a0378b9 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:53.994191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 960/1932] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:53 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:53.998592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fa6ff-6bc8-4396-952d-38f3feb3688e admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:54.013245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fa6ff-6bc8-4396-952d-38f3feb3688e admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:54.019883 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff5155-3bbd-4512-a0f9-29788035c8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:54.021319 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff5155-3bbd-4512-a0f9-29788035c8da admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:54.021609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff5155-3bbd-4512-a0f9-29788035c8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:54.021887 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff5155-3bbd-4512-a0f9-29788035c8da admin admin] PATH_INFO: `/v3/projects/8d40c028e25545988168d5ea96e9c395/users/adbdf37c958243af9b97e7cc03058877/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:54.041968 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff5155-3bbd-4512-a0f9-29788035c8da admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d40c028e25545988168d5ea96e9c395, user_id=adbdf37c958243af9b97e7cc03058877, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:54.045125 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff5155-3bbd-4512-a0f9-29788035c8da admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:54.151178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 973/1933] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:53 2022] PUT /identity/v3/projects/8d40c028e25545988168d5ea96e9c395/users/adbdf37c958243af9b97e7cc03058877/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:54.166751 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df8f76-f70e-4152-ad16-45404ee8c060 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:54.167189 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df8f76-f70e-4152-ad16-45404ee8c060 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:54.167586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df8f76-f70e-4152-ad16-45404ee8c060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:54.219863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-e2df8f76-f70e-4152-ad16-45404ee8c060 None None] MFA Rules not processed for user `adbdf37c958243af9b97e7cc03058877`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:54.232156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e2df8f76-f70e-4152-ad16-45404ee8c060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:54.296566 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:54.357504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 961/1934] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:54.386139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff5155-3bbd-4512-a0f9-29788035c8da admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:54.388718 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e5ff5155-3bbd-4512-a0f9-29788035c8da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:54.470516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff5155-3bbd-4512-a0f9-29788035c8da admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:54.474466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cffc4-0558-4b17-9c74-6c1ac41694e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:54.475075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cffc4-0558-4b17-9c74-6c1ac41694e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:54.475320 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cffc4-0558-4b17-9c74-6c1ac41694e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:54.475598 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cffc4-0558-4b17-9c74-6c1ac41694e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:54.485090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cffc4-0558-4b17-9c74-6c1ac41694e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:54.489529 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cffc4-0558-4b17-9c74-6c1ac41694e1 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:54.499806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 974/1935] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:56.668716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df8f76-f70e-4152-ad16-45404ee8c060 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:56.671726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e2df8f76-f70e-4152-ad16-45404ee8c060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:56.745955 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cffc4-0558-4b17-9c74-6c1ac41694e1 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:56.747456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9b9cffc4-0558-4b17-9c74-6c1ac41694e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:56.814002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df8f76-f70e-4152-ad16-45404ee8c060 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:56.825238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a2171-560b-4a17-9330-394ea25ad350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:56.826818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a2171-560b-4a17-9330-394ea25ad350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:56.827183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a2171-560b-4a17-9330-394ea25ad350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:56.828244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a2171-560b-4a17-9330-394ea25ad350 admin admin] PATH_INFO: `/v3/users/ea7ab7126db3408ab2858f6fcb3d4f20` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:56.832187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a2171-560b-4a17-9330-394ea25ad350 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea7ab7126db3408ab2858f6fcb3d4f20)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:56.834935 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a2171-560b-4a17-9330-394ea25ad350 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:56.888825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cffc4-0558-4b17-9c74-6c1ac41694e1 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:56.896653 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d1ea6-5738-4f9c-9af9-336ba1458587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:56.897733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d1ea6-5738-4f9c-9af9-336ba1458587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:56.898095 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d1ea6-5738-4f9c-9af9-336ba1458587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:56.898500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d1ea6-5738-4f9c-9af9-336ba1458587 admin admin] PATH_INFO: `/v3/users/575156e97837437596b3f8711ba12484` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:56.899944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d1ea6-5738-4f9c-9af9-336ba1458587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=575156e97837437596b3f8711ba12484)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:56.910261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d1ea6-5738-4f9c-9af9-336ba1458587 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:57.087978 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-fb5a2171-560b-4a17-9330-394ea25ad350 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea7ab7126db3408ab2858f6fcb3d4f20'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:57.184553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cc8d1ea6-5738-4f9c-9af9-336ba1458587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '575156e97837437596b3f8711ba12484'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:57.190809 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:57.198772 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-fb5a2171-560b-4a17-9330-394ea25ad350 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea7ab7126db3408ab2858f6fcb3d4f20'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:57.203707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-fb5a2171-560b-4a17-9330-394ea25ad350 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea7ab7126db3408ab2858f6fcb3d4f20'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:57.203707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-fb5a2171-560b-4a17-9330-394ea25ad350 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea7ab7126db3408ab2858f6fcb3d4f20'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:57.204462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cc8d1ea6-5738-4f9c-9af9-336ba1458587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '575156e97837437596b3f8711ba12484'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:57.221126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-fb5a2171-560b-4a17-9330-394ea25ad350 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea7ab7126db3408ab2858f6fcb3d4f20'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:57.237817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 962/1936] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:56 2022] DELETE /identity/v3/users/ea7ab7126db3408ab2858f6fcb3d4f20 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:57.242204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b4736-b4f9-4933-9728-eaa45e6dfa5b None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:57.242489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b4736-b4f9-4933-9728-eaa45e6dfa5b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:57.242794 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b4736-b4f9-4933-9728-eaa45e6dfa5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:57.261856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-480b4736-b4f9-4933-9728-eaa45e6dfa5b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:57.263072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-480b4736-b4f9-4933-9728-eaa45e6dfa5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:57.282769 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cc8d1ea6-5738-4f9c-9af9-336ba1458587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '575156e97837437596b3f8711ba12484'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:57.291879 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cc8d1ea6-5738-4f9c-9af9-336ba1458587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '575156e97837437596b3f8711ba12484'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:57.292595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cc8d1ea6-5738-4f9c-9af9-336ba1458587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '575156e97837437596b3f8711ba12484'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:57.300352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 975/1937] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:56 2022] DELETE /identity/v3/users/575156e97837437596b3f8711ba12484 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:57.304917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655289a3-8127-4d79-a574-5551cfad39f1 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:57.305336 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655289a3-8127-4d79-a574-5551cfad39f1 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:57.305766 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655289a3-8127-4d79-a574-5551cfad39f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:57.316131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:57.325108 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-655289a3-8127-4d79-a574-5551cfad39f1 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:57.326335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-655289a3-8127-4d79-a574-5551cfad39f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:57.337612 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 963/1938] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:57.350109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b4736-b4f9-4933-9728-eaa45e6dfa5b None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:57.351621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-480b4736-b4f9-4933-9728-eaa45e6dfa5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:57.409637 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 976/1939] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:57.419506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b4736-b4f9-4933-9728-eaa45e6dfa5b None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:57.435499 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655289a3-8127-4d79-a574-5551cfad39f1 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:57.437384 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-655289a3-8127-4d79-a574-5551cfad39f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:57.501330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d6101-bedb-4419-ba4f-9d585d6ebbb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:57.502272 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d6101-bedb-4419-ba4f-9d585d6ebbb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:57.502665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d6101-bedb-4419-ba4f-9d585d6ebbb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:57.503021 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d6101-bedb-4419-ba4f-9d585d6ebbb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:57.515501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d6101-bedb-4419-ba4f-9d585d6ebbb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:57.516091 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d6101-bedb-4419-ba4f-9d585d6ebbb3 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:57.537497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655289a3-8127-4d79-a574-5551cfad39f1 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:57.537497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3aeef-b4b1-424b-86a4-e92f2da58167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:57.537497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3aeef-b4b1-424b-86a4-e92f2da58167 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:57.537497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3aeef-b4b1-424b-86a4-e92f2da58167 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:57.538739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3aeef-b4b1-424b-86a4-e92f2da58167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:57.545107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 964/1940] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:57.546792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3aeef-b4b1-424b-86a4-e92f2da58167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:57.548845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3aeef-b4b1-424b-86a4-e92f2da58167 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:57.559316 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 977/1941] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:22:58.207459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d6101-bedb-4419-ba4f-9d585d6ebbb3 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:58.207459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-845d6101-bedb-4419-ba4f-9d585d6ebbb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:58.208280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3aeef-b4b1-424b-86a4-e92f2da58167 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:58.209914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-20e3aeef-b4b1-424b-86a4-e92f2da58167 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:58.226615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d6101-bedb-4419-ba4f-9d585d6ebbb3 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:58.232078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:58.234192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:58.234424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:58.234892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3aeef-b4b1-424b-86a4-e92f2da58167 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:58.235005 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6 admin admin] PATH_INFO: `/v3/projects/58d527df216e44dfa1a89fca94d8fda3` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:58.236791 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58d527df216e44dfa1a89fca94d8fda3)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:58.241047 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa695f-c908-48ec-987f-741a676dad7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:58.241446 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:58.242725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa695f-c908-48ec-987f-741a676dad7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:58.250108 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa695f-c908-48ec-987f-741a676dad7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:58.255311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa695f-c908-48ec-987f-741a676dad7f admin admin] PATH_INFO: `/v3/projects/358a43481125457cb095446b7daa9bef` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:58.255311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa695f-c908-48ec-987f-741a676dad7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=358a43481125457cb095446b7daa9bef)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:58.255311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa695f-c908-48ec-987f-741a676dad7f admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:58.367536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58d527df216e44dfa1a89fca94d8fda3', 'request_id': 'req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:58.376599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58d527df216e44dfa1a89fca94d8fda3', 'request_id': 'req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:58.431641 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6afa695f-c908-48ec-987f-741a676dad7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '358a43481125457cb095446b7daa9bef', 'request_id': 'req-6afa695f-c908-48ec-987f-741a676dad7f'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:58.439407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6afa695f-c908-48ec-987f-741a676dad7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '358a43481125457cb095446b7daa9bef', 'request_id': 'req-6afa695f-c908-48ec-987f-741a676dad7f'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:58.446599 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6afa695f-c908-48ec-987f-741a676dad7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '358a43481125457cb095446b7daa9bef', 'request_id': 'req-6afa695f-c908-48ec-987f-741a676dad7f'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:58.451375 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:58.476985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58d527df216e44dfa1a89fca94d8fda3', 'request_id': 'req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:58.489305 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6 admin admin] The token cache is being invalidate because project 58d527df216e44dfa1a89fca94d8fda3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:58.489512 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:58.490146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:58.491706 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 965/1942] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:58 2022] DELETE /identity/v3/projects/58d527df216e44dfa1a89fca94d8fda3 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:58.496106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:58.497475 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:58.550332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:58.571033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6afa695f-c908-48ec-987f-741a676dad7f admin admin] The token cache is being invalidate because project 358a43481125457cb095446b7daa9bef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:22:58.571305 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6afa695f-c908-48ec-987f-741a676dad7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:58.573091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6afa695f-c908-48ec-987f-741a676dad7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:58.576630 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 978/1943] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:58 2022] DELETE /identity/v3/projects/358a43481125457cb095446b7daa9bef => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:58.583805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778953f-d55d-41af-a7a8-7904ae7dc4e9 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:58.584176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778953f-d55d-41af-a7a8-7904ae7dc4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:58.584349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778953f-d55d-41af-a7a8-7904ae7dc4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:58.630324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-7778953f-d55d-41af-a7a8-7904ae7dc4e9 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:58.632349 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9bb8a-26f4-4c78-acd1-dc4edac5aaa6 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:58.632756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7778953f-d55d-41af-a7a8-7904ae7dc4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:58.750414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d63e25-714d-4616-9054-24c314385dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:58.750414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d63e25-714d-4616-9054-24c314385dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:58.750414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d63e25-714d-4616-9054-24c314385dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:58.750414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d63e25-714d-4616-9054-24c314385dde admin admin] PATH_INFO: `/v3/users/842c30e419334f769592f9d1382e78cf` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:58.751462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d63e25-714d-4616-9054-24c314385dde admin admin] RBAC: Authorizing `identity:delete_user(user_id=842c30e419334f769592f9d1382e78cf)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:58.752069 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d63e25-714d-4616-9054-24c314385dde admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:58.763717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 979/1944] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:58.772006 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2fd68-ad52-47ce-bcc1-db3d22c8a89c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:58.772006 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2fd68-ad52-47ce-bcc1-db3d22c8a89c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:58.772276 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2fd68-ad52-47ce-bcc1-db3d22c8a89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:58.793519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-bcd2fd68-ad52-47ce-bcc1-db3d22c8a89c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:58.794891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-bcd2fd68-ad52-47ce-bcc1-db3d22c8a89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:58.806926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 980/1945] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:22:58.811105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2fd68-ad52-47ce-bcc1-db3d22c8a89c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:58.830736 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2fd68-ad52-47ce-bcc1-db3d22c8a89c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:58.843652 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d8a1c-697d-436e-915e-63ffcff40771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:58.844432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d8a1c-697d-436e-915e-63ffcff40771 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:58.845004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d8a1c-697d-436e-915e-63ffcff40771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:58.845374 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d8a1c-697d-436e-915e-63ffcff40771 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:58.846165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d8a1c-697d-436e-915e-63ffcff40771 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:22:58.846480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d8a1c-697d-436e-915e-63ffcff40771 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:58.848286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d8a1c-697d-436e-915e-63ffcff40771 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:58.866109 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 981/1946] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:22:58.871057 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d8a1c-697d-436e-915e-63ffcff40771 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:58.888003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d8a1c-697d-436e-915e-63ffcff40771 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:58.893542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57460f27-9286-411d-9d09-a4c1117a57d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:58.894662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57460f27-9286-411d-9d09-a4c1117a57d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:58.894952 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57460f27-9286-411d-9d09-a4c1117a57d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:58.895413 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57460f27-9286-411d-9d09-a4c1117a57d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:58.896276 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57460f27-9286-411d-9d09-a4c1117a57d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:58.898264 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57460f27-9286-411d-9d09-a4c1117a57d3 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:59.017548 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 982/1947] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:22:58 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:59.021519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57460f27-9286-411d-9d09-a4c1117a57d3 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:59.029142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-18d63e25-714d-4616-9054-24c314385dde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '842c30e419334f769592f9d1382e78cf'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:59.107982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57460f27-9286-411d-9d09-a4c1117a57d3 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:59.114236 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a406c2b-aa46-4590-b44a-dd12ff4fc8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:59.115022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a406c2b-aa46-4590-b44a-dd12ff4fc8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:59.115370 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a406c2b-aa46-4590-b44a-dd12ff4fc8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:59.115715 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a406c2b-aa46-4590-b44a-dd12ff4fc8c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:59.116478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a406c2b-aa46-4590-b44a-dd12ff4fc8c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:59.118557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a406c2b-aa46-4590-b44a-dd12ff4fc8c7 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:59.140267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-18d63e25-714d-4616-9054-24c314385dde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '842c30e419334f769592f9d1382e78cf'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:59.142051 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-18d63e25-714d-4616-9054-24c314385dde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '842c30e419334f769592f9d1382e78cf'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:59.149178 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-18d63e25-714d-4616-9054-24c314385dde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '842c30e419334f769592f9d1382e78cf'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:59.158219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-18d63e25-714d-4616-9054-24c314385dde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '842c30e419334f769592f9d1382e78cf'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:22:59.174657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 966/1948] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:22:58 2022] DELETE /identity/v3/users/842c30e419334f769592f9d1382e78cf => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:22:59.252138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 983/1949] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:22:59 2022] POST /identity/v3/users => generated 287 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:22:59.258878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d63e25-714d-4616-9054-24c314385dde admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:59.260587 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-18d63e25-714d-4616-9054-24c314385dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:59.322462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:59.337058 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d63e25-714d-4616-9054-24c314385dde admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:59.342065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbaeea-fc88-4830-bd20-6bd7ca14d014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:59.342873 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbaeea-fc88-4830-bd20-6bd7ca14d014 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:59.343182 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbaeea-fc88-4830-bd20-6bd7ca14d014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:59.343532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbaeea-fc88-4830-bd20-6bd7ca14d014 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:59.345044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbaeea-fc88-4830-bd20-6bd7ca14d014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:59.347290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbaeea-fc88-4830-bd20-6bd7ca14d014 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:59.357950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 967/1950] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:22:59.362309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a406c2b-aa46-4590-b44a-dd12ff4fc8c7 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:59.373777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a406c2b-aa46-4590-b44a-dd12ff4fc8c7 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:59.379973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5e225-8727-49b3-bb3d-c969a5fc044a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:59.380908 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5e225-8727-49b3-bb3d-c969a5fc044a admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:59.381235 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5e225-8727-49b3-bb3d-c969a5fc044a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:59.381518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5e225-8727-49b3-bb3d-c969a5fc044a admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:59.382338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5e225-8727-49b3-bb3d-c969a5fc044a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:59.383978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5e225-8727-49b3-bb3d-c969a5fc044a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:59.396002 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 984/1951] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:22:59.401165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbaeea-fc88-4830-bd20-6bd7ca14d014 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:59.415962 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbaeea-fc88-4830-bd20-6bd7ca14d014 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:59.422358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ea039-2dfd-4287-94f3-cacb86dc7a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:59.423382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ea039-2dfd-4287-94f3-cacb86dc7a94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:59.423731 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ea039-2dfd-4287-94f3-cacb86dc7a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:59.424127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573ea039-2dfd-4287-94f3-cacb86dc7a94 admin admin] PATH_INFO: `/v3/projects/e73242c46698496ba8b80ce681417938/users/1d4be529ee5e4765ada8eaee33e21c64/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:59.442402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ea039-2dfd-4287-94f3-cacb86dc7a94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e73242c46698496ba8b80ce681417938, user_id=1d4be529ee5e4765ada8eaee33e21c64, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:59.445466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573ea039-2dfd-4287-94f3-cacb86dc7a94 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:59.514026 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 968/1952] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:22:59 2022] PUT /identity/v3/projects/e73242c46698496ba8b80ce681417938/users/1d4be529ee5e4765ada8eaee33e21c64/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:22:59.519667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9552a-b3d7-4d3e-b7fb-43b0e4fd5dac None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:59.519980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9552a-b3d7-4d3e-b7fb-43b0e4fd5dac None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:59.520249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9552a-b3d7-4d3e-b7fb-43b0e4fd5dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:59.543980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-01b9552a-b3d7-4d3e-b7fb-43b0e4fd5dac None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:22:59.545291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-01b9552a-b3d7-4d3e-b7fb-43b0e4fd5dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:59.578234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:22:59.612013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 985/1953] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:22:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:22:59.622923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ea039-2dfd-4287-94f3-cacb86dc7a94 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:59.624797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-573ea039-2dfd-4287-94f3-cacb86dc7a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:59.697544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573ea039-2dfd-4287-94f3-cacb86dc7a94 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:59.701182 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db221baf-a3d2-4890-8304-2971fdb23c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:59.701836 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db221baf-a3d2-4890-8304-2971fdb23c9e service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:59.702115 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db221baf-a3d2-4890-8304-2971fdb23c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:59.702391 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db221baf-a3d2-4890-8304-2971fdb23c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:59.712335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db221baf-a3d2-4890-8304-2971fdb23c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:59.717951 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db221baf-a3d2-4890-8304-2971fdb23c9e service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:22:59.725164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 969/1954] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:22:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:22:59.965605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9552a-b3d7-4d3e-b7fb-43b0e4fd5dac None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:22:59.967369 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-01b9552a-b3d7-4d3e-b7fb-43b0e4fd5dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:22:59.981678 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9552a-b3d7-4d3e-b7fb-43b0e4fd5dac None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:22:59.988278 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d96f8-e1d0-4068-a199-931c9cb43d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:22:59.994052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d96f8-e1d0-4068-a199-931c9cb43d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:22:59.994052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d96f8-e1d0-4068-a199-931c9cb43d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:22:59.994052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d96f8-e1d0-4068-a199-931c9cb43d53 admin admin] PATH_INFO: `/v3/projects/1a594c9158a44ce4af323ba690b544c5` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:22:59.994052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d96f8-e1d0-4068-a199-931c9cb43d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a594c9158a44ce4af323ba690b544c5)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:22:59.994052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d96f8-e1d0-4068-a199-931c9cb43d53 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:00.161403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-828d96f8-e1d0-4068-a199-931c9cb43d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a594c9158a44ce4af323ba690b544c5', 'request_id': 'req-828d96f8-e1d0-4068-a199-931c9cb43d53'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:00.168333 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-828d96f8-e1d0-4068-a199-931c9cb43d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a594c9158a44ce4af323ba690b544c5', 'request_id': 'req-828d96f8-e1d0-4068-a199-931c9cb43d53'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:00.175337 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-828d96f8-e1d0-4068-a199-931c9cb43d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a594c9158a44ce4af323ba690b544c5', 'request_id': 'req-828d96f8-e1d0-4068-a199-931c9cb43d53'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:00.306911 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-828d96f8-e1d0-4068-a199-931c9cb43d53 admin admin] The token cache is being invalidate because project 1a594c9158a44ce4af323ba690b544c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:23:00.307574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-828d96f8-e1d0-4068-a199-931c9cb43d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:00.308019 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-828d96f8-e1d0-4068-a199-931c9cb43d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:00.309843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 986/1955] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:22:59 2022] DELETE /identity/v3/projects/1a594c9158a44ce4af323ba690b544c5 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:00.314021 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db221baf-a3d2-4890-8304-2971fdb23c9e service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:00.324994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db221baf-a3d2-4890-8304-2971fdb23c9e service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:00.391864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2d2ad-3911-40d8-82a9-3fcf8dc0671b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:00.392778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2d2ad-3911-40d8-82a9-3fcf8dc0671b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:00.393098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2d2ad-3911-40d8-82a9-3fcf8dc0671b admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:00.393386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2d2ad-3911-40d8-82a9-3fcf8dc0671b admin admin] PATH_INFO: `/v3/users/18a90c9c793a4f1687f46d082c6d96ae` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:00.394683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2d2ad-3911-40d8-82a9-3fcf8dc0671b admin admin] RBAC: Authorizing `identity:delete_user(user_id=18a90c9c793a4f1687f46d082c6d96ae)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:00.396623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2d2ad-3911-40d8-82a9-3fcf8dc0671b admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:00.615702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-2cf2d2ad-3911-40d8-82a9-3fcf8dc0671b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18a90c9c793a4f1687f46d082c6d96ae'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:00.708287 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-2cf2d2ad-3911-40d8-82a9-3fcf8dc0671b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18a90c9c793a4f1687f46d082c6d96ae'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:00.709346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-2cf2d2ad-3911-40d8-82a9-3fcf8dc0671b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18a90c9c793a4f1687f46d082c6d96ae'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:00.710655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-2cf2d2ad-3911-40d8-82a9-3fcf8dc0671b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18a90c9c793a4f1687f46d082c6d96ae'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:00.727395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:00.732113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-2cf2d2ad-3911-40d8-82a9-3fcf8dc0671b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18a90c9c793a4f1687f46d082c6d96ae'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:00.758415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 970/1956] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:23:00 2022] DELETE /identity/v3/users/18a90c9c793a4f1687f46d082c6d96ae => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:23:01.405269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d96f8-e1d0-4068-a199-931c9cb43d53 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:01.407386 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-828d96f8-e1d0-4068-a199-931c9cb43d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:01.425326 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d96f8-e1d0-4068-a199-931c9cb43d53 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:01.546630 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3e548-d8c5-4012-913d-84311ea000ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:01.548314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3e548-d8c5-4012-913d-84311ea000ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:01.548614 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3e548-d8c5-4012-913d-84311ea000ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:01.548895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3e548-d8c5-4012-913d-84311ea000ef admin admin] PATH_INFO: `/v3/projects/cda77967154e49b6aeb6f90e64a3e31c` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:01.550510 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3e548-d8c5-4012-913d-84311ea000ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=cda77967154e49b6aeb6f90e64a3e31c)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:01.558935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3e548-d8c5-4012-913d-84311ea000ef admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:01.644985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8bdf2-c8ab-4e06-9218-e4336874e21a None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:01.645592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8bdf2-c8ab-4e06-9218-e4336874e21a None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:01.646632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8bdf2-c8ab-4e06-9218-e4336874e21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:01.692033 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-d8e8bdf2-c8ab-4e06-9218-e4336874e21a None None] MFA Rules not processed for user `1d4be529ee5e4765ada8eaee33e21c64`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:01.693603 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d8e8bdf2-c8ab-4e06-9218-e4336874e21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:01.740288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c7a3e548-d8c5-4012-913d-84311ea000ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda77967154e49b6aeb6f90e64a3e31c', 'request_id': 'req-c7a3e548-d8c5-4012-913d-84311ea000ef'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:01.748332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c7a3e548-d8c5-4012-913d-84311ea000ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda77967154e49b6aeb6f90e64a3e31c', 'request_id': 'req-c7a3e548-d8c5-4012-913d-84311ea000ef'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:01.753457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:01.757173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c7a3e548-d8c5-4012-913d-84311ea000ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda77967154e49b6aeb6f90e64a3e31c', 'request_id': 'req-c7a3e548-d8c5-4012-913d-84311ea000ef'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:01.783161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 971/1957] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:01.794375 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8bdf2-c8ab-4e06-9218-e4336874e21a None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:01.795843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d8e8bdf2-c8ab-4e06-9218-e4336874e21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:01.856407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8bdf2-c8ab-4e06-9218-e4336874e21a None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:01.862094 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c7a3e548-d8c5-4012-913d-84311ea000ef admin admin] The token cache is being invalidate because project cda77967154e49b6aeb6f90e64a3e31c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:23:01.862393 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c7a3e548-d8c5-4012-913d-84311ea000ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:01.863054 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c7a3e548-d8c5-4012-913d-84311ea000ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:01.864652 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 987/1958] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:23:01 2022] DELETE /identity/v3/projects/cda77967154e49b6aeb6f90e64a3e31c => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:23:01.869765 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd9dac-1f50-4da9-bdbb-b8234ef364ac None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:01.870076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd9dac-1f50-4da9-bdbb-b8234ef364ac None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:01.870363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd9dac-1f50-4da9-bdbb-b8234ef364ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:01.891390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-20cd9dac-1f50-4da9-bdbb-b8234ef364ac None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:01.893274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-20cd9dac-1f50-4da9-bdbb-b8234ef364ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:01.914320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bceb8-0a83-4511-999d-7b1637e37176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:01.915002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bceb8-0a83-4511-999d-7b1637e37176 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:01.915316 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bceb8-0a83-4511-999d-7b1637e37176 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:01.915596 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bceb8-0a83-4511-999d-7b1637e37176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:01.917517 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f69bceb8-0a83-4511-999d-7b1637e37176 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:01.957856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 988/1959] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:01.966307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1080377-e738-4dc4-9a43-dc609b197c4f None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:01.966610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1080377-e738-4dc4-9a43-dc609b197c4f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:01.966890 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1080377-e738-4dc4-9a43-dc609b197c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:01.974933 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:01.984071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bceb8-0a83-4511-999d-7b1637e37176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:01.986802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bceb8-0a83-4511-999d-7b1637e37176 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:01.987083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-f1080377-e738-4dc4-9a43-dc609b197c4f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:01.988299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f1080377-e738-4dc4-9a43-dc609b197c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:01.997128 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 989/1960] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:23:01.997601 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 972/1961] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:23:02.002459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1080377-e738-4dc4-9a43-dc609b197c4f None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:02.012615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1080377-e738-4dc4-9a43-dc609b197c4f None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:02.017673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834bfd5-d190-47be-a439-9077ca80e271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:02.018345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834bfd5-d190-47be-a439-9077ca80e271 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:02.018632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834bfd5-d190-47be-a439-9077ca80e271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:02.018915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834bfd5-d190-47be-a439-9077ca80e271 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:02.019657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834bfd5-d190-47be-a439-9077ca80e271 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:23:02.019964 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834bfd5-d190-47be-a439-9077ca80e271 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:02.021698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834bfd5-d190-47be-a439-9077ca80e271 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:02.035176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 990/1962] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:23:02.039650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bceb8-0a83-4511-999d-7b1637e37176 service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:02.052323 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bceb8-0a83-4511-999d-7b1637e37176 service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:02.058312 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0394f33-0676-41a1-8a19-b9d482368113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:02.059003 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0394f33-0676-41a1-8a19-b9d482368113 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:02.059290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0394f33-0676-41a1-8a19-b9d482368113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:02.059570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0394f33-0676-41a1-8a19-b9d482368113 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:02.060281 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0394f33-0676-41a1-8a19-b9d482368113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:02.062324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0394f33-0676-41a1-8a19-b9d482368113 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:02.155223 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 973/1963] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:23:02 2022] POST /identity/v3/projects => generated 367 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:23:02.160232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834bfd5-d190-47be-a439-9077ca80e271 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:02.170646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834bfd5-d190-47be-a439-9077ca80e271 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:02.232455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ddf00-c2e5-4038-8fb5-f166cabaa7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:02.233163 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ddf00-c2e5-4038-8fb5-f166cabaa7bd admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:02.233443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ddf00-c2e5-4038-8fb5-f166cabaa7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:02.233721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ddf00-c2e5-4038-8fb5-f166cabaa7bd admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:02.236314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ddf00-c2e5-4038-8fb5-f166cabaa7bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:02.239372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ddf00-c2e5-4038-8fb5-f166cabaa7bd admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:02.336794 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 991/1964] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:23:02 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:23:02.341170 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0394f33-0676-41a1-8a19-b9d482368113 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:02.354041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0394f33-0676-41a1-8a19-b9d482368113 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:02.358931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed81d6-6e31-4e8f-9803-e7b5e4f61697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:02.359641 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed81d6-6e31-4e8f-9803-e7b5e4f61697 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:02.360108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed81d6-6e31-4e8f-9803-e7b5e4f61697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:02.360377 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed81d6-6e31-4e8f-9803-e7b5e4f61697 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:02.361152 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed81d6-6e31-4e8f-9803-e7b5e4f61697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:02.362759 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed81d6-6e31-4e8f-9803-e7b5e4f61697 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:02.373690 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 974/1965] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:02.378251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ddf00-c2e5-4038-8fb5-f166cabaa7bd admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:02.399064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ddf00-c2e5-4038-8fb5-f166cabaa7bd admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:02.408432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32c881-ab86-4df5-abc0-e696aaef9960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:02.409529 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32c881-ab86-4df5-abc0-e696aaef9960 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:02.409640 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32c881-ab86-4df5-abc0-e696aaef9960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:02.409874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32c881-ab86-4df5-abc0-e696aaef9960 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:02.411321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32c881-ab86-4df5-abc0-e696aaef9960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:02.413160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32c881-ab86-4df5-abc0-e696aaef9960 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:02.425598 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 992/1966] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:02.431250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed81d6-6e31-4e8f-9803-e7b5e4f61697 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:02.442726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed81d6-6e31-4e8f-9803-e7b5e4f61697 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:02.448279 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979cfe9-94f4-4c0a-91e2-65d32305b7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:02.449240 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979cfe9-94f4-4c0a-91e2-65d32305b7e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:02.451440 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979cfe9-94f4-4c0a-91e2-65d32305b7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:02.451752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979cfe9-94f4-4c0a-91e2-65d32305b7e4 admin admin] PATH_INFO: `/v3/projects/b2f44e31260240558eab4143262c3978/users/b21628dc64aa447d8234ef7848adbb18/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:02.473125 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979cfe9-94f4-4c0a-91e2-65d32305b7e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2f44e31260240558eab4143262c3978, user_id=b21628dc64aa447d8234ef7848adbb18, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:02.475749 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979cfe9-94f4-4c0a-91e2-65d32305b7e4 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:02.549114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 975/1967] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:23:02 2022] PUT /identity/v3/projects/b2f44e31260240558eab4143262c3978/users/b21628dc64aa447d8234ef7848adbb18/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:23:02.557634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9b537-cd66-4bdd-b276-2732bf4085c1 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:02.558000 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9b537-cd66-4bdd-b276-2732bf4085c1 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:02.558377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e9b537-cd66-4bdd-b276-2732bf4085c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:02.587669 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-43e9b537-cd66-4bdd-b276-2732bf4085c1 None None] MFA Rules not processed for user `b21628dc64aa447d8234ef7848adbb18`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:02.590826 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-43e9b537-cd66-4bdd-b276-2732bf4085c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:02.626162 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:02.683100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 993/1968] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:02.690574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979cfe9-94f4-4c0a-91e2-65d32305b7e4 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:02.692624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d979cfe9-94f4-4c0a-91e2-65d32305b7e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:02.783638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979cfe9-94f4-4c0a-91e2-65d32305b7e4 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:02.787541 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a42cc-1979-41d1-938e-15af6a4d2065 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fbf33189aa94f4f973d558d615a23e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:02.788215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a42cc-1979-41d1-938e-15af6a4d2065 service glance] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:02.788579 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a42cc-1979-41d1-938e-15af6a4d2065 service glance] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:02.793381 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a42cc-1979-41d1-938e-15af6a4d2065 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:02.804328 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a42cc-1979-41d1-938e-15af6a4d2065 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:02.809635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a42cc-1979-41d1-938e-15af6a4d2065 service glance] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:02.818715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 976/1969] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:23:04.886853 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bf4b6-50c2-4333-997a-c59780eb3e0d None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:04.890827 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bf4b6-50c2-4333-997a-c59780eb3e0d None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:04.891961 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bf4b6-50c2-4333-997a-c59780eb3e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:04.923935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-780bf4b6-50c2-4333-997a-c59780eb3e0d None None] MFA Rules not processed for user `1d4be529ee5e4765ada8eaee33e21c64`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:04.933151 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-780bf4b6-50c2-4333-997a-c59780eb3e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:05.011401 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 994/1970] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:10.640721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a42cc-1979-41d1-938e-15af6a4d2065 service glance] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:10.642714 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-8f2a42cc-1979-41d1-938e-15af6a4d2065 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:10.712137 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a42cc-1979-41d1-938e-15af6a4d2065 service glance] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:10.716666 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e15f0-eabc-4a13-96e2-79748555cdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:10.717634 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e15f0-eabc-4a13-96e2-79748555cdd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:10.717920 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e15f0-eabc-4a13-96e2-79748555cdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:10.718193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e15f0-eabc-4a13-96e2-79748555cdd4 admin admin] PATH_INFO: `/v3/users/adbdf37c958243af9b97e7cc03058877` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:10.719929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e15f0-eabc-4a13-96e2-79748555cdd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adbdf37c958243af9b97e7cc03058877)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:10.722089 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12e15f0-eabc-4a13-96e2-79748555cdd4 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:10.930875 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:10.943708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f12e15f0-eabc-4a13-96e2-79748555cdd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adbdf37c958243af9b97e7cc03058877'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:11.054496 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f12e15f0-eabc-4a13-96e2-79748555cdd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adbdf37c958243af9b97e7cc03058877'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:11.055156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f12e15f0-eabc-4a13-96e2-79748555cdd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adbdf37c958243af9b97e7cc03058877'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:11.055800 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f12e15f0-eabc-4a13-96e2-79748555cdd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adbdf37c958243af9b97e7cc03058877'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:11.068927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f12e15f0-eabc-4a13-96e2-79748555cdd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adbdf37c958243af9b97e7cc03058877'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:11.082253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 977/1971] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:23:10 2022] DELETE /identity/v3/users/adbdf37c958243af9b97e7cc03058877 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:11.086904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e98e15-5ae7-4c15-9387-48d83035c0fb None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:11.087451 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e98e15-5ae7-4c15-9387-48d83035c0fb None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:11.087451 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e98e15-5ae7-4c15-9387-48d83035c0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:11.106101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-82e98e15-5ae7-4c15-9387-48d83035c0fb None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:11.107324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-82e98e15-5ae7-4c15-9387-48d83035c0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:11.179208 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 995/1972] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:11.189865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e15f0-eabc-4a13-96e2-79748555cdd4 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:11.191352 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f12e15f0-eabc-4a13-96e2-79748555cdd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:11.256597 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e15f0-eabc-4a13-96e2-79748555cdd4 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:11.261111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8adaf3b-2ba5-4f0a-b594-a83194064aea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:11.261777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adaf3b-2ba5-4f0a-b594-a83194064aea service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:11.262092 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adaf3b-2ba5-4f0a-b594-a83194064aea service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:11.262345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8adaf3b-2ba5-4f0a-b594-a83194064aea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:11.279178 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8adaf3b-2ba5-4f0a-b594-a83194064aea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:11.280655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8adaf3b-2ba5-4f0a-b594-a83194064aea service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:11.292376 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 978/1973] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:12.063775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e98e15-5ae7-4c15-9387-48d83035c0fb None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:12.065513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-82e98e15-5ae7-4c15-9387-48d83035c0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:12.106556 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e98e15-5ae7-4c15-9387-48d83035c0fb None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:12.114561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ac30a-3daa-43c4-b88e-68c09082925d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:12.115456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ac30a-3daa-43c4-b88e-68c09082925d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:12.121557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ac30a-3daa-43c4-b88e-68c09082925d admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:12.121557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ac30a-3daa-43c4-b88e-68c09082925d admin admin] PATH_INFO: `/v3/projects/8d40c028e25545988168d5ea96e9c395` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:12.121557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ac30a-3daa-43c4-b88e-68c09082925d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d40c028e25545988168d5ea96e9c395)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:12.121557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ac30a-3daa-43c4-b88e-68c09082925d admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:12.241643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cd1ac30a-3daa-43c4-b88e-68c09082925d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d40c028e25545988168d5ea96e9c395', 'request_id': 'req-cd1ac30a-3daa-43c4-b88e-68c09082925d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:12.248204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cd1ac30a-3daa-43c4-b88e-68c09082925d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d40c028e25545988168d5ea96e9c395', 'request_id': 'req-cd1ac30a-3daa-43c4-b88e-68c09082925d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:12.256693 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cd1ac30a-3daa-43c4-b88e-68c09082925d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d40c028e25545988168d5ea96e9c395', 'request_id': 'req-cd1ac30a-3daa-43c4-b88e-68c09082925d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:12.317868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cd1ac30a-3daa-43c4-b88e-68c09082925d admin admin] The token cache is being invalidate because project 8d40c028e25545988168d5ea96e9c395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:23:12.318139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cd1ac30a-3daa-43c4-b88e-68c09082925d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:12.318801 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cd1ac30a-3daa-43c4-b88e-68c09082925d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:12.320712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 996/1974] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:23:12 2022] DELETE /identity/v3/projects/8d40c028e25545988168d5ea96e9c395 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:12.325656 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffe485-9169-4d2f-b217-f0061672fa83 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:12.326166 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffe485-9169-4d2f-b217-f0061672fa83 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:12.326435 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffe485-9169-4d2f-b217-f0061672fa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:12.350955 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-9fffe485-9169-4d2f-b217-f0061672fa83 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:12.352350 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9fffe485-9169-4d2f-b217-f0061672fa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:12.396685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fefdc6-46d6-44c9-8a11-53662d3081b6 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:12.397274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fefdc6-46d6-44c9-8a11-53662d3081b6 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:12.397552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fefdc6-46d6-44c9-8a11-53662d3081b6 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:12.398988 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 997/1975] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jun 15 10:23:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:23:12.406463 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e2455-7e76-48b7-9b0a-336a39438921 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:12.406763 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e2455-7e76-48b7-9b0a-336a39438921 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:12.407409 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e2455-7e76-48b7-9b0a-336a39438921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:12.425140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:12.439757 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-687e2455-7e76-48b7-9b0a-336a39438921 None None] MFA Rules not processed for user `12899a7a59b54e3199b277ff94e30d42`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:12.441341 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-687e2455-7e76-48b7-9b0a-336a39438921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:12.451670 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 979/1976] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:12.460426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f577d2-07dc-4f0b-b46e-4125e8a78485 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:12.460718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f577d2-07dc-4f0b-b46e-4125e8a78485 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:12.461053 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f577d2-07dc-4f0b-b46e-4125e8a78485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:12.480509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-e3f577d2-07dc-4f0b-b46e-4125e8a78485 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:12.481764 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e3f577d2-07dc-4f0b-b46e-4125e8a78485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:12.494129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 980/1977] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:23:12.499779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f577d2-07dc-4f0b-b46e-4125e8a78485 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:12.513923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f577d2-07dc-4f0b-b46e-4125e8a78485 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:12.519688 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d6d81-cc86-4d2f-b590-8820f31d2742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:12.520366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d6d81-cc86-4d2f-b590-8820f31d2742 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:12.520673 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d6d81-cc86-4d2f-b590-8820f31d2742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:12.520973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d6d81-cc86-4d2f-b590-8820f31d2742 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:12.521716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d6d81-cc86-4d2f-b590-8820f31d2742 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:23:12.522475 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d6d81-cc86-4d2f-b590-8820f31d2742 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:12.524149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d6d81-cc86-4d2f-b590-8820f31d2742 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:12.529557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 998/1978] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jun 15 10:23:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:12.553578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e2455-7e76-48b7-9b0a-336a39438921 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:12.555069 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-687e2455-7e76-48b7-9b0a-336a39438921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:12.559335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 981/1979] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:23:12.560481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d6d81-cc86-4d2f-b590-8820f31d2742 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:12.565526 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-984d6d81-cc86-4d2f-b590-8820f31d2742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:12.574224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e2455-7e76-48b7-9b0a-336a39438921 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:12.580380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d6d81-cc86-4d2f-b590-8820f31d2742 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:12.581482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddea00-aecb-42be-aa7d-d367f87b787b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:12.582149 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddea00-aecb-42be-aa7d-d367f87b787b service placement] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:12.582434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddea00-aecb-42be-aa7d-d367f87b787b service placement] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:12.582712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ddea00-aecb-42be-aa7d-d367f87b787b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:12.585597 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60bed12-4c6c-4b6b-9ff7-6653cb646c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:12.586472 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bed12-4c6c-4b6b-9ff7-6653cb646c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:12.586759 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bed12-4c6c-4b6b-9ff7-6653cb646c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:12.587037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bed12-4c6c-4b6b-9ff7-6653cb646c07 admin admin] PATH_INFO: `/v3/users/3f8028df66d44c478268a84167c9ff19` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:12.588459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60bed12-4c6c-4b6b-9ff7-6653cb646c07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f8028df66d44c478268a84167c9ff19)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:12.590508 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60bed12-4c6c-4b6b-9ff7-6653cb646c07 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:12.591977 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ddea00-aecb-42be-aa7d-d367f87b787b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:12.595340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ddea00-aecb-42be-aa7d-d367f87b787b service placement] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:12.604432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 999/1980] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:23:12.605303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddea00-aecb-42be-aa7d-d367f87b787b service placement] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:12.617776 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ddea00-aecb-42be-aa7d-d367f87b787b service placement] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:12.624021 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d22ae-616f-4a68-84d6-60a268e3c8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:12.624734 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d22ae-616f-4a68-84d6-60a268e3c8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:12.625051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d22ae-616f-4a68-84d6-60a268e3c8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:12.625404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d22ae-616f-4a68-84d6-60a268e3c8d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:12.626789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75d22ae-616f-4a68-84d6-60a268e3c8d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:12.628990 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75d22ae-616f-4a68-84d6-60a268e3c8d3 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:12.767718 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1000/1981] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:23:12 2022] POST /identity/v3/projects => generated 389 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:23:12.771845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d22ae-616f-4a68-84d6-60a268e3c8d3 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:12.785240 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:12.838339 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d22ae-616f-4a68-84d6-60a268e3c8d3 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:12.843724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f883c48-9d81-4551-ae56-626a241085f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:12.844530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f883c48-9d81-4551-ae56-626a241085f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:12.845903 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f883c48-9d81-4551-ae56-626a241085f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:12.845903 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f883c48-9d81-4551-ae56-626a241085f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:12.846894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f883c48-9d81-4551-ae56-626a241085f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:12.849181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f883c48-9d81-4551-ae56-626a241085f0 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:12.886247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e60bed12-4c6c-4b6b-9ff7-6653cb646c07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f8028df66d44c478268a84167c9ff19'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:12.949174 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1001/1982] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:23:12 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:23:12.953295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f883c48-9d81-4551-ae56-626a241085f0 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:12.964941 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e60bed12-4c6c-4b6b-9ff7-6653cb646c07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f8028df66d44c478268a84167c9ff19'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:12.965530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f883c48-9d81-4551-ae56-626a241085f0 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:12.970993 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e60bed12-4c6c-4b6b-9ff7-6653cb646c07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f8028df66d44c478268a84167c9ff19'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:12.971849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e60bed12-4c6c-4b6b-9ff7-6653cb646c07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f8028df66d44c478268a84167c9ff19'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:12.972407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0666409-ede7-4b56-923f-fad8eb84d53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:12.972985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0666409-ede7-4b56-923f-fad8eb84d53a admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:12.973288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0666409-ede7-4b56-923f-fad8eb84d53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:12.973566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0666409-ede7-4b56-923f-fad8eb84d53a admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:12.974426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0666409-ede7-4b56-923f-fad8eb84d53a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:12.976121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0666409-ede7-4b56-923f-fad8eb84d53a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:12.990144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1002/1983] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:23:12.990353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e60bed12-4c6c-4b6b-9ff7-6653cb646c07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f8028df66d44c478268a84167c9ff19'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:12.991662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0666409-ede7-4b56-923f-fad8eb84d53a admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:12.993918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e0666409-ede7-4b56-923f-fad8eb84d53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:13.004899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 982/1984] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:23:12 2022] DELETE /identity/v3/users/3f8028df66d44c478268a84167c9ff19 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:13.008144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0666409-ede7-4b56-923f-fad8eb84d53a admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:13.054209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:13.069580 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bbb61c-6f9f-409f-accc-d2d997614885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:13.070257 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbb61c-6f9f-409f-accc-d2d997614885 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:13.070540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbb61c-6f9f-409f-accc-d2d997614885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:13.071673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bbb61c-6f9f-409f-accc-d2d997614885 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:13.072808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbb61c-6f9f-409f-accc-d2d997614885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:13.074218 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bbb61c-6f9f-409f-accc-d2d997614885 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:13.089286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1003/1985] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:13.090897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60bed12-4c6c-4b6b-9ff7-6653cb646c07 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:13.102696 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60bed12-4c6c-4b6b-9ff7-6653cb646c07 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:13.111383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c13c2-34cc-490d-b7c6-03bb13839837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:13.112344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c13c2-34cc-490d-b7c6-03bb13839837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:13.112629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c13c2-34cc-490d-b7c6-03bb13839837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:13.112910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c13c2-34cc-490d-b7c6-03bb13839837 admin admin] PATH_INFO: `/v3/projects/cb9ef70beebf460ca603c258705bfa69/users/179d85a2dc7744b79b7111e4a0a20256/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:13.130200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c13c2-34cc-490d-b7c6-03bb13839837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb9ef70beebf460ca603c258705bfa69, user_id=179d85a2dc7744b79b7111e4a0a20256, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:13.132927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c13c2-34cc-490d-b7c6-03bb13839837 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:13.241724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 983/1986] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:23:13 2022] PUT /identity/v3/projects/cb9ef70beebf460ca603c258705bfa69/users/179d85a2dc7744b79b7111e4a0a20256/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:23:13.257658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052a5c4-ac10-4760-b3a0-0523f3056aa8 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:13.258270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052a5c4-ac10-4760-b3a0-0523f3056aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:13.258513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052a5c4-ac10-4760-b3a0-0523f3056aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:13.293850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-1052a5c4-ac10-4760-b3a0-0523f3056aa8 None None] MFA Rules not processed for user `179d85a2dc7744b79b7111e4a0a20256`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:13.295121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-1052a5c4-ac10-4760-b3a0-0523f3056aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:13.385529 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1004/1987] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:13.393983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c13c2-34cc-490d-b7c6-03bb13839837 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:13.396439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-650c13c2-34cc-490d-b7c6-03bb13839837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:13.400372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052a5c4-ac10-4760-b3a0-0523f3056aa8 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:13.401976 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-1052a5c4-ac10-4760-b3a0-0523f3056aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:13.484474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052a5c4-ac10-4760-b3a0-0523f3056aa8 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:13.488651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848d698-52c9-4955-a9b7-4b99229ea918 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:13.489340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848d698-52c9-4955-a9b7-4b99229ea918 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:13.489622 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848d698-52c9-4955-a9b7-4b99229ea918 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:13.489920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848d698-52c9-4955-a9b7-4b99229ea918 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:13.499081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848d698-52c9-4955-a9b7-4b99229ea918 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:13.500644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848d698-52c9-4955-a9b7-4b99229ea918 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:13.502004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c13c2-34cc-490d-b7c6-03bb13839837 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:13.507529 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f63fb-7f86-4168-beb1-47925fa3acb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:13.508442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f63fb-7f86-4168-beb1-47925fa3acb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:13.508723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f63fb-7f86-4168-beb1-47925fa3acb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:13.509047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f63fb-7f86-4168-beb1-47925fa3acb4 admin admin] PATH_INFO: `/v3/projects/0b8e3aa03dcf4c5a989a604548e9c4ba` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:13.510361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f63fb-7f86-4168-beb1-47925fa3acb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b8e3aa03dcf4c5a989a604548e9c4ba)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:13.510913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1005/1988] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:23:13.512434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f63fb-7f86-4168-beb1-47925fa3acb4 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:13.637376 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-446f63fb-7f86-4168-beb1-47925fa3acb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b8e3aa03dcf4c5a989a604548e9c4ba', 'request_id': 'req-446f63fb-7f86-4168-beb1-47925fa3acb4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:13.645160 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-446f63fb-7f86-4168-beb1-47925fa3acb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b8e3aa03dcf4c5a989a604548e9c4ba', 'request_id': 'req-446f63fb-7f86-4168-beb1-47925fa3acb4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:13.769293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-446f63fb-7f86-4168-beb1-47925fa3acb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b8e3aa03dcf4c5a989a604548e9c4ba', 'request_id': 'req-446f63fb-7f86-4168-beb1-47925fa3acb4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:13.781581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-446f63fb-7f86-4168-beb1-47925fa3acb4 admin admin] The token cache is being invalidate because project 0b8e3aa03dcf4c5a989a604548e9c4ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:23:13.781881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-446f63fb-7f86-4168-beb1-47925fa3acb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:13.782574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-446f63fb-7f86-4168-beb1-47925fa3acb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:13.784202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 984/1989] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:23:13 2022] DELETE /identity/v3/projects/0b8e3aa03dcf4c5a989a604548e9c4ba => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:13.789231 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767d7b3-767e-45e3-9cd9-7b28a0a3dc31 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:13.789516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767d7b3-767e-45e3-9cd9-7b28a0a3dc31 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:13.789792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767d7b3-767e-45e3-9cd9-7b28a0a3dc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:13.808293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-3767d7b3-767e-45e3-9cd9-7b28a0a3dc31 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:13.809529 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-3767d7b3-767e-45e3-9cd9-7b28a0a3dc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:13.879918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1006/1990] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:13.887758 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e0cb8-ccc9-48a7-96ef-634386ee0d8b None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:13.887948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e0cb8-ccc9-48a7-96ef-634386ee0d8b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:13.888226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e0cb8-ccc9-48a7-96ef-634386ee0d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:13.908441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-d10e0cb8-ccc9-48a7-96ef-634386ee0d8b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:13.909725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d10e0cb8-ccc9-48a7-96ef-634386ee0d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:13.917866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 985/1991] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:13.932303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767d7b3-767e-45e3-9cd9-7b28a0a3dc31 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:13.944171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767d7b3-767e-45e3-9cd9-7b28a0a3dc31 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:13.948885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10135c26-8cdb-43ed-ac87-aba39e251333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:13.949715 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10135c26-8cdb-43ed-ac87-aba39e251333 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:13.949998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10135c26-8cdb-43ed-ac87-aba39e251333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:13.950275 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10135c26-8cdb-43ed-ac87-aba39e251333 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:13.951016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10135c26-8cdb-43ed-ac87-aba39e251333 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:23:13.951325 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10135c26-8cdb-43ed-ac87-aba39e251333 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:13.952991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10135c26-8cdb-43ed-ac87-aba39e251333 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:13.963164 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:13.967233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1007/1992] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:23:13.971531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e0cb8-ccc9-48a7-96ef-634386ee0d8b None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:13.977274 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:13.981228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e0cb8-ccc9-48a7-96ef-634386ee0d8b None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:13.986553 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bce846-67e2-41d7-94eb-b3923907cc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:13.987210 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bce846-67e2-41d7-94eb-b3923907cc66 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:13.987492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bce846-67e2-41d7-94eb-b3923907cc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:13.990116 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bce846-67e2-41d7-94eb-b3923907cc66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:13.990116 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bce846-67e2-41d7-94eb-b3923907cc66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:13.990520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bce846-67e2-41d7-94eb-b3923907cc66 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:14.098338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 986/1993] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:23:13 2022] POST /identity/v3/projects => generated 401 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:23:14.102271 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10135c26-8cdb-43ed-ac87-aba39e251333 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:14.114105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10135c26-8cdb-43ed-ac87-aba39e251333 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:14.174328 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac72f70-4d50-4165-81dc-60eb662da1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:14.174905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac72f70-4d50-4165-81dc-60eb662da1b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:14.175186 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac72f70-4d50-4165-81dc-60eb662da1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:14.175462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac72f70-4d50-4165-81dc-60eb662da1b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:14.176157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac72f70-4d50-4165-81dc-60eb662da1b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:14.178122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac72f70-4d50-4165-81dc-60eb662da1b0 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:14.287089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1008/1994] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:23:14 2022] POST /identity/v3/users => generated 301 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:23:14.291781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bce846-67e2-41d7-94eb-b3923907cc66 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:14.303699 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bce846-67e2-41d7-94eb-b3923907cc66 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:14.308773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd65082-47c5-4aa8-857b-44bb74d58cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:14.310190 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd65082-47c5-4aa8-857b-44bb74d58cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:14.310190 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd65082-47c5-4aa8-857b-44bb74d58cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:14.310190 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd65082-47c5-4aa8-857b-44bb74d58cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:14.310782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd65082-47c5-4aa8-857b-44bb74d58cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:14.312406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd65082-47c5-4aa8-857b-44bb74d58cc6 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:14.326711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 987/1995] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:14.331971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac72f70-4d50-4165-81dc-60eb662da1b0 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:14.347510 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac72f70-4d50-4165-81dc-60eb662da1b0 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:14.354516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00822cb9-d94a-4dc8-98b5-da69b6b8b65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:14.363154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00822cb9-d94a-4dc8-98b5-da69b6b8b65d admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:14.363618 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00822cb9-d94a-4dc8-98b5-da69b6b8b65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:14.364014 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00822cb9-d94a-4dc8-98b5-da69b6b8b65d admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:14.364939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00822cb9-d94a-4dc8-98b5-da69b6b8b65d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:14.369157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00822cb9-d94a-4dc8-98b5-da69b6b8b65d admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:14.383619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1009/1996] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:14.387749 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd65082-47c5-4aa8-857b-44bb74d58cc6 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:14.399139 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd65082-47c5-4aa8-857b-44bb74d58cc6 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:14.404572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b4487-3556-4ed8-9b44-3c534f4a58cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:14.405812 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b4487-3556-4ed8-9b44-3c534f4a58cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:14.406109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b4487-3556-4ed8-9b44-3c534f4a58cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:14.406392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b4487-3556-4ed8-9b44-3c534f4a58cc admin admin] PATH_INFO: `/v3/projects/17d5f565d02b49f4afd7de265bd1fd56/users/f5aa7f1514024d25ae3609dcb59f2a6c/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:14.429303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b4487-3556-4ed8-9b44-3c534f4a58cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=17d5f565d02b49f4afd7de265bd1fd56, user_id=f5aa7f1514024d25ae3609dcb59f2a6c, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:14.432841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b4487-3556-4ed8-9b44-3c534f4a58cc admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:14.536615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 988/1997] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:23:14 2022] PUT /identity/v3/projects/17d5f565d02b49f4afd7de265bd1fd56/users/f5aa7f1514024d25ae3609dcb59f2a6c/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:14.550052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74fa1f-1f5e-4f09-bedf-c3c0f5e6ed7f None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:14.550349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74fa1f-1f5e-4f09-bedf-c3c0f5e6ed7f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:14.550660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c74fa1f-1f5e-4f09-bedf-c3c0f5e6ed7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:14.578725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-0c74fa1f-1f5e-4f09-bedf-c3c0f5e6ed7f None None] MFA Rules not processed for user `f5aa7f1514024d25ae3609dcb59f2a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:14.580074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0c74fa1f-1f5e-4f09-bedf-c3c0f5e6ed7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:14.650960 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1010/1998] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:14.655502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b4487-3556-4ed8-9b44-3c534f4a58cc admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:14.722062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b4487-3556-4ed8-9b44-3c534f4a58cc admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:14.726829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee6d57-efc0-4de9-8bf4-f112a0a23a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:14.727491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee6d57-efc0-4de9-8bf4-f112a0a23a9a admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:14.727777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee6d57-efc0-4de9-8bf4-f112a0a23a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:14.728057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee6d57-efc0-4de9-8bf4-f112a0a23a9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:14.728762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee6d57-efc0-4de9-8bf4-f112a0a23a9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:14.731215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee6d57-efc0-4de9-8bf4-f112a0a23a9a admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:14.855943 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 989/1999] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:23:14 2022] POST /identity/v3/projects => generated 399 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:23:14.860807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74fa1f-1f5e-4f09-bedf-c3c0f5e6ed7f None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:14.871581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c74fa1f-1f5e-4f09-bedf-c3c0f5e6ed7f None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:14.937449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93113b3e-a8d5-4f83-be0a-932477267e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:14.938150 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93113b3e-a8d5-4f83-be0a-932477267e03 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:14.939010 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93113b3e-a8d5-4f83-be0a-932477267e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:14.939010 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93113b3e-a8d5-4f83-be0a-932477267e03 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:14.939597 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93113b3e-a8d5-4f83-be0a-932477267e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:14.941540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93113b3e-a8d5-4f83-be0a-932477267e03 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:15.028360 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:15.035130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1011/2000] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:23:14 2022] POST /identity/v3/users => generated 306 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:23:15.038977 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee6d57-efc0-4de9-8bf4-f112a0a23a9a admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:15.050002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee6d57-efc0-4de9-8bf4-f112a0a23a9a admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:15.054707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355803f-3da0-48e5-90e5-72c3cc6bcfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:15.055365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355803f-3da0-48e5-90e5-72c3cc6bcfce admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:15.055646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355803f-3da0-48e5-90e5-72c3cc6bcfce admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:15.055920 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355803f-3da0-48e5-90e5-72c3cc6bcfce admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:15.056657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355803f-3da0-48e5-90e5-72c3cc6bcfce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:15.058284 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355803f-3da0-48e5-90e5-72c3cc6bcfce admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:15.070245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 990/2001] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:15.075190 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93113b3e-a8d5-4f83-be0a-932477267e03 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:15.086466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93113b3e-a8d5-4f83-be0a-932477267e03 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:15.091154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6409e-7013-40ec-bf85-c67dbca93457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:15.092029 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6409e-7013-40ec-bf85-c67dbca93457 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:15.092304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6409e-7013-40ec-bf85-c67dbca93457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:15.092573 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6409e-7013-40ec-bf85-c67dbca93457 admin admin] PATH_INFO: `/v3/projects/ed769fc21aed4601a7f11a7482d33e0b/users/a3dfb470f09d4fcb8dea5d8f7c5fb6e6/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:15.109738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6409e-7013-40ec-bf85-c67dbca93457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed769fc21aed4601a7f11a7482d33e0b, user_id=a3dfb470f09d4fcb8dea5d8f7c5fb6e6, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:15.112855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6409e-7013-40ec-bf85-c67dbca93457 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:15.275463 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1012/2002] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:23:15 2022] PUT /identity/v3/projects/ed769fc21aed4601a7f11a7482d33e0b/users/a3dfb470f09d4fcb8dea5d8f7c5fb6e6/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:15.279434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355803f-3da0-48e5-90e5-72c3cc6bcfce admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:15.291268 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355803f-3da0-48e5-90e5-72c3cc6bcfce admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:15.367062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb1904b-3b9f-45bf-92d4-35915b287385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:15.367814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1904b-3b9f-45bf-92d4-35915b287385 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:15.368110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1904b-3b9f-45bf-92d4-35915b287385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:15.368389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb1904b-3b9f-45bf-92d4-35915b287385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:15.369277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1904b-3b9f-45bf-92d4-35915b287385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:15.370942 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb1904b-3b9f-45bf-92d4-35915b287385 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:15.383110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 991/2003] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:15.387017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6409e-7013-40ec-bf85-c67dbca93457 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:15.398776 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6409e-7013-40ec-bf85-c67dbca93457 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:15.403905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014590b1-84f4-4a64-ab41-4d0c5949dbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:15.404805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014590b1-84f4-4a64-ab41-4d0c5949dbab admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:15.405124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014590b1-84f4-4a64-ab41-4d0c5949dbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:15.405486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014590b1-84f4-4a64-ab41-4d0c5949dbab admin admin] PATH_INFO: `/v3/projects/ed769fc21aed4601a7f11a7482d33e0b/users/a3dfb470f09d4fcb8dea5d8f7c5fb6e6/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:15.407482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014590b1-84f4-4a64-ab41-4d0c5949dbab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed769fc21aed4601a7f11a7482d33e0b, user_id=a3dfb470f09d4fcb8dea5d8f7c5fb6e6, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:15.410740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014590b1-84f4-4a64-ab41-4d0c5949dbab admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:15.421252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1013/2004] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:23:15 2022] PUT /identity/v3/projects/ed769fc21aed4601a7f11a7482d33e0b/users/a3dfb470f09d4fcb8dea5d8f7c5fb6e6/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:15.431678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ee636-e4d5-48b4-920c-c4f73c0304d3 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:15.432109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ee636-e4d5-48b4-920c-c4f73c0304d3 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:15.432229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ee636-e4d5-48b4-920c-c4f73c0304d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:15.462878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-904ee636-e4d5-48b4-920c-c4f73c0304d3 None None] MFA Rules not processed for user `a3dfb470f09d4fcb8dea5d8f7c5fb6e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:15.464319 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-904ee636-e4d5-48b4-920c-c4f73c0304d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:15.475123 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:15.567089 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 992/2005] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:15 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:15.579990 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014590b1-84f4-4a64-ab41-4d0c5949dbab admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:15.581901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-014590b1-84f4-4a64-ab41-4d0c5949dbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:15.655936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014590b1-84f4-4a64-ab41-4d0c5949dbab admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:15.660365 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5b0ab-de56-4292-83a9-d07ef7b61aaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:15.660942 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5b0ab-de56-4292-83a9-d07ef7b61aaa service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:15.661249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5b0ab-de56-4292-83a9-d07ef7b61aaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:15.661544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5b0ab-de56-4292-83a9-d07ef7b61aaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:15.672137 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5b0ab-de56-4292-83a9-d07ef7b61aaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:15.674339 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5b0ab-de56-4292-83a9-d07ef7b61aaa service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:15.684951 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1014/2006] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:15 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:15.889746 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ee636-e4d5-48b4-920c-c4f73c0304d3 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:15.901671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ee636-e4d5-48b4-920c-c4f73c0304d3 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:15.905333 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1e025-376c-4e94-a787-47ee93fdd610 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:15.905988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1e025-376c-4e94-a787-47ee93fdd610 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:15.906272 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1e025-376c-4e94-a787-47ee93fdd610 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:15.906548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1e025-376c-4e94-a787-47ee93fdd610 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:15.964550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1e025-376c-4e94-a787-47ee93fdd610 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:15.966149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1e025-376c-4e94-a787-47ee93fdd610 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:15.975881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 993/2007] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:25.680031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=42,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:25.971643 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=54 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:26.437404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5b0ab-de56-4292-83a9-d07ef7b61aaa service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:26.507262 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5b0ab-de56-4292-83a9-d07ef7b61aaa service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:26.511914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dc82c-6774-4187-a563-ed735695d559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:26.512807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dc82c-6774-4187-a563-ed735695d559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:26.515637 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dc82c-6774-4187-a563-ed735695d559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:26.515637 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dc82c-6774-4187-a563-ed735695d559 admin admin] PATH_INFO: `/v3/users/f5aa7f1514024d25ae3609dcb59f2a6c` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:26.515637 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742dc82c-6774-4187-a563-ed735695d559 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5aa7f1514024d25ae3609dcb59f2a6c)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:26.517265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742dc82c-6774-4187-a563-ed735695d559 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:26.749299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-742dc82c-6774-4187-a563-ed735695d559 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5aa7f1514024d25ae3609dcb59f2a6c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:26.762405 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-742dc82c-6774-4187-a563-ed735695d559 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5aa7f1514024d25ae3609dcb59f2a6c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:26.843105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-742dc82c-6774-4187-a563-ed735695d559 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5aa7f1514024d25ae3609dcb59f2a6c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:26.852438 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-742dc82c-6774-4187-a563-ed735695d559 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5aa7f1514024d25ae3609dcb59f2a6c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:26.857166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-742dc82c-6774-4187-a563-ed735695d559 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5aa7f1514024d25ae3609dcb59f2a6c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:26.859061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1015/2008] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:23:26 2022] DELETE /identity/v3/users/f5aa7f1514024d25ae3609dcb59f2a6c => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:26.863864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a950bb-ecf6-475e-b98c-462189c2f8df None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:26.864152 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a950bb-ecf6-475e-b98c-462189c2f8df None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:26.864428 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a950bb-ecf6-475e-b98c-462189c2f8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:26.883131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a5a950bb-ecf6-475e-b98c-462189c2f8df None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:26.884329 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a5a950bb-ecf6-475e-b98c-462189c2f8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:26.952086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 994/2009] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:26.962649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dc82c-6774-4187-a563-ed735695d559 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:26.964084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-742dc82c-6774-4187-a563-ed735695d559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:27.025243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dc82c-6774-4187-a563-ed735695d559 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:27.028790 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0063426-66bc-4f81-b655-56fd15573e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:27.029477 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0063426-66bc-4f81-b655-56fd15573e81 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:27.029769 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0063426-66bc-4f81-b655-56fd15573e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:27.030048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0063426-66bc-4f81-b655-56fd15573e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:27.040244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0063426-66bc-4f81-b655-56fd15573e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:27.041718 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0063426-66bc-4f81-b655-56fd15573e81 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:27.051369 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1016/2010] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:23:27.511164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a950bb-ecf6-475e-b98c-462189c2f8df None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:27.512620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a5a950bb-ecf6-475e-b98c-462189c2f8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:27.528925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a950bb-ecf6-475e-b98c-462189c2f8df None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:27.535509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f2e8e-fcda-4a64-86a9-913f18350801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:27.540395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f2e8e-fcda-4a64-86a9-913f18350801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:27.540395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f2e8e-fcda-4a64-86a9-913f18350801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:27.540395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f2e8e-fcda-4a64-86a9-913f18350801 admin admin] PATH_INFO: `/v3/projects/17d5f565d02b49f4afd7de265bd1fd56` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:27.548817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f2e8e-fcda-4a64-86a9-913f18350801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17d5f565d02b49f4afd7de265bd1fd56)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:27.548817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f2e8e-fcda-4a64-86a9-913f18350801 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:27.718031 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e84f2e8e-fcda-4a64-86a9-913f18350801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17d5f565d02b49f4afd7de265bd1fd56', 'request_id': 'req-e84f2e8e-fcda-4a64-86a9-913f18350801'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:27.724813 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e84f2e8e-fcda-4a64-86a9-913f18350801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17d5f565d02b49f4afd7de265bd1fd56', 'request_id': 'req-e84f2e8e-fcda-4a64-86a9-913f18350801'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:27.797516 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e84f2e8e-fcda-4a64-86a9-913f18350801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17d5f565d02b49f4afd7de265bd1fd56', 'request_id': 'req-e84f2e8e-fcda-4a64-86a9-913f18350801'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:27.808757 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e84f2e8e-fcda-4a64-86a9-913f18350801 admin admin] The token cache is being invalidate because project 17d5f565d02b49f4afd7de265bd1fd56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:23:27.809067 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e84f2e8e-fcda-4a64-86a9-913f18350801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:27.809709 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e84f2e8e-fcda-4a64-86a9-913f18350801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:27.811285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 995/2011] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:23:27 2022] DELETE /identity/v3/projects/17d5f565d02b49f4afd7de265bd1fd56 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:27.815938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0063426-66bc-4f81-b655-56fd15573e81 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:27.826549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0063426-66bc-4f81-b655-56fd15573e81 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:27.846671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:27.887040 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe3b60-2068-4e63-a06d-c8edacbc0d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:27.887945 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe3b60-2068-4e63-a06d-c8edacbc0d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:27.888228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe3b60-2068-4e63-a06d-c8edacbc0d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:27.888502 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe3b60-2068-4e63-a06d-c8edacbc0d55 admin admin] PATH_INFO: `/v3/users/a3dfb470f09d4fcb8dea5d8f7c5fb6e6` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:27.889938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe3b60-2068-4e63-a06d-c8edacbc0d55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3dfb470f09d4fcb8dea5d8f7c5fb6e6)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:27.892461 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe3b60-2068-4e63-a06d-c8edacbc0d55 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:28.047209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-8afe3b60-2068-4e63-a06d-c8edacbc0d55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3dfb470f09d4fcb8dea5d8f7c5fb6e6'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:28.059282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-8afe3b60-2068-4e63-a06d-c8edacbc0d55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3dfb470f09d4fcb8dea5d8f7c5fb6e6'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:28.119548 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-8afe3b60-2068-4e63-a06d-c8edacbc0d55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3dfb470f09d4fcb8dea5d8f7c5fb6e6'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:28.128472 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-8afe3b60-2068-4e63-a06d-c8edacbc0d55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3dfb470f09d4fcb8dea5d8f7c5fb6e6'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:28.129178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-8afe3b60-2068-4e63-a06d-c8edacbc0d55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3dfb470f09d4fcb8dea5d8f7c5fb6e6'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:28.135104 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1017/2012] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:23:27 2022] DELETE /identity/v3/users/a3dfb470f09d4fcb8dea5d8f7c5fb6e6 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:28.572128 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f2e8e-fcda-4a64-86a9-913f18350801 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:28.573686 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e84f2e8e-fcda-4a64-86a9-913f18350801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:28.589600 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f2e8e-fcda-4a64-86a9-913f18350801 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:28.655722 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf39ef0-c853-4ebc-be0e-42ceabc19ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:28.656604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf39ef0-c853-4ebc-be0e-42ceabc19ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:28.656950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf39ef0-c853-4ebc-be0e-42ceabc19ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:28.657266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf39ef0-c853-4ebc-be0e-42ceabc19ba5 admin admin] PATH_INFO: `/v3/projects/ed769fc21aed4601a7f11a7482d33e0b` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:28.660188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf39ef0-c853-4ebc-be0e-42ceabc19ba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed769fc21aed4601a7f11a7482d33e0b)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:28.662896 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf39ef0-c853-4ebc-be0e-42ceabc19ba5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:28.795290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:28.812873 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-0bf39ef0-c853-4ebc-be0e-42ceabc19ba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed769fc21aed4601a7f11a7482d33e0b', 'request_id': 'req-0bf39ef0-c853-4ebc-be0e-42ceabc19ba5'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:28.819788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-0bf39ef0-c853-4ebc-be0e-42ceabc19ba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed769fc21aed4601a7f11a7482d33e0b', 'request_id': 'req-0bf39ef0-c853-4ebc-be0e-42ceabc19ba5'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:28.895082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-0bf39ef0-c853-4ebc-be0e-42ceabc19ba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed769fc21aed4601a7f11a7482d33e0b', 'request_id': 'req-0bf39ef0-c853-4ebc-be0e-42ceabc19ba5'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:28.906509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-0bf39ef0-c853-4ebc-be0e-42ceabc19ba5 admin admin] The token cache is being invalidate because project ed769fc21aed4601a7f11a7482d33e0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:23:28.906801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-0bf39ef0-c853-4ebc-be0e-42ceabc19ba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:28.907456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-0bf39ef0-c853-4ebc-be0e-42ceabc19ba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:28.909665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 996/2013] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:23:28 2022] DELETE /identity/v3/projects/ed769fc21aed4601a7f11a7482d33e0b => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:28.914647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82589724-5b35-43a3-b844-7a928d1ac32e None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:28.915135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82589724-5b35-43a3-b844-7a928d1ac32e None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:28.915243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82589724-5b35-43a3-b844-7a928d1ac32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:28.934390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-82589724-5b35-43a3-b844-7a928d1ac32e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:28.935607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-82589724-5b35-43a3-b844-7a928d1ac32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:28.997127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1018/2014] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:29.005792 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e46539-c188-41e4-a41b-6f55686dad76 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:29.006014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e46539-c188-41e4-a41b-6f55686dad76 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:29.006244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e46539-c188-41e4-a41b-6f55686dad76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:29.025742 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a8e46539-c188-41e4-a41b-6f55686dad76 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:29.027311 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a8e46539-c188-41e4-a41b-6f55686dad76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:29.036069 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 997/2015] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:23:29.040700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82589724-5b35-43a3-b844-7a928d1ac32e None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:29.051808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82589724-5b35-43a3-b844-7a928d1ac32e None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:29.056493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed527a2-e4d5-4c07-a9f3-12a330d833be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:29.057210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed527a2-e4d5-4c07-a9f3-12a330d833be admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:29.057498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed527a2-e4d5-4c07-a9f3-12a330d833be admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:29.057779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed527a2-e4d5-4c07-a9f3-12a330d833be admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:29.058574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed527a2-e4d5-4c07-a9f3-12a330d833be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:23:29.058882 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed527a2-e4d5-4c07-a9f3-12a330d833be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:29.060510 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed527a2-e4d5-4c07-a9f3-12a330d833be admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:29.072864 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1019/2016] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:23:29.077315 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e46539-c188-41e4-a41b-6f55686dad76 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:29.087643 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e46539-c188-41e4-a41b-6f55686dad76 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:29.094206 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3e9df-c9a4-4a96-835d-c0d012f4d4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:29.094206 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3e9df-c9a4-4a96-835d-c0d012f4d4ec admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:29.094206 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3e9df-c9a4-4a96-835d-c0d012f4d4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:29.094206 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3e9df-c9a4-4a96-835d-c0d012f4d4ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:29.094599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3e9df-c9a4-4a96-835d-c0d012f4d4ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:29.097173 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3e9df-c9a4-4a96-835d-c0d012f4d4ec admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:29.224346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 998/2017] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:23:29 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:23:29.228655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed527a2-e4d5-4c07-a9f3-12a330d833be admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:29.238966 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed527a2-e4d5-4c07-a9f3-12a330d833be admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:29.266426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:29.298323 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce87f5b-00d2-42b2-a929-a30d341f35c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:29.299254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce87f5b-00d2-42b2-a929-a30d341f35c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:29.299537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce87f5b-00d2-42b2-a929-a30d341f35c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:29.299819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce87f5b-00d2-42b2-a929-a30d341f35c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:29.300520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce87f5b-00d2-42b2-a929-a30d341f35c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:29.302531 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce87f5b-00d2-42b2-a929-a30d341f35c4 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:29.405675 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1020/2018] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:23:29 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:23:29.409941 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3e9df-c9a4-4a96-835d-c0d012f4d4ec admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:29.420862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3e9df-c9a4-4a96-835d-c0d012f4d4ec admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:29.425942 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ed076-1431-466c-8036-0eb9f2876a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:29.426672 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ed076-1431-466c-8036-0eb9f2876a1c admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:29.426984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ed076-1431-466c-8036-0eb9f2876a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:29.427267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ed076-1431-466c-8036-0eb9f2876a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:29.428018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ed076-1431-466c-8036-0eb9f2876a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:29.429751 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ed076-1431-466c-8036-0eb9f2876a1c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:29.441330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 999/2019] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:23:29.445743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce87f5b-00d2-42b2-a929-a30d341f35c4 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:29.456546 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce87f5b-00d2-42b2-a929-a30d341f35c4 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:29.461299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e71866-1e42-4e18-843d-a496a98f6a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:29.461970 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e71866-1e42-4e18-843d-a496a98f6a81 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:29.462289 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e71866-1e42-4e18-843d-a496a98f6a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:29.462576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e71866-1e42-4e18-843d-a496a98f6a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:29.463321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e71866-1e42-4e18-843d-a496a98f6a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:29.464931 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e71866-1e42-4e18-843d-a496a98f6a81 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:29.475805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1021/2020] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:29.480147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ed076-1431-466c-8036-0eb9f2876a1c admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:29.491121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ed076-1431-466c-8036-0eb9f2876a1c admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:29.495738 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35451d6b-0cf2-437c-84d5-de68501f4163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:29.496691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35451d6b-0cf2-437c-84d5-de68501f4163 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:29.496992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35451d6b-0cf2-437c-84d5-de68501f4163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:29.497178 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35451d6b-0cf2-437c-84d5-de68501f4163 admin admin] PATH_INFO: `/v3/projects/32b28e20bdd54f09a7977fce5db00d2d/users/64e9366e183741a498eeb06834fcf99c/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:29.513944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35451d6b-0cf2-437c-84d5-de68501f4163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32b28e20bdd54f09a7977fce5db00d2d, user_id=64e9366e183741a498eeb06834fcf99c, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:29.516640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35451d6b-0cf2-437c-84d5-de68501f4163 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:29.613347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1000/2021] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:23:29 2022] PUT /identity/v3/projects/32b28e20bdd54f09a7977fce5db00d2d/users/64e9366e183741a498eeb06834fcf99c/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:29.624584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb861a6-776e-4a78-bf78-852a06db66e3 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:29.627157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb861a6-776e-4a78-bf78-852a06db66e3 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:29.627157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb861a6-776e-4a78-bf78-852a06db66e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:29.653858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-6fb861a6-776e-4a78-bf78-852a06db66e3 None None] MFA Rules not processed for user `64e9366e183741a498eeb06834fcf99c`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:29.655077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6fb861a6-776e-4a78-bf78-852a06db66e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:29.722904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1022/2022] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:29.727451 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35451d6b-0cf2-437c-84d5-de68501f4163 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:29.790769 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35451d6b-0cf2-437c-84d5-de68501f4163 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:29.795587 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c51d83-46b0-4eb0-8701-539557fb765e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:29.796249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c51d83-46b0-4eb0-8701-539557fb765e admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:29.796536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c51d83-46b0-4eb0-8701-539557fb765e admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:29.796815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c51d83-46b0-4eb0-8701-539557fb765e admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:29.797549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c51d83-46b0-4eb0-8701-539557fb765e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:29.799993 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c51d83-46b0-4eb0-8701-539557fb765e admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:29.890956 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1001/2023] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:23:29 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:23:29.895367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb861a6-776e-4a78-bf78-852a06db66e3 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:29.906667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb861a6-776e-4a78-bf78-852a06db66e3 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:29.969596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b36d7-6884-454b-9090-ab9583518074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:29.970303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b36d7-6884-454b-9090-ab9583518074 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:29.970628 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b36d7-6884-454b-9090-ab9583518074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:29.970865 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b36d7-6884-454b-9090-ab9583518074 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:29.971579 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b36d7-6884-454b-9090-ab9583518074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:29.973572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b36d7-6884-454b-9090-ab9583518074 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:30.097973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1023/2024] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:23:29 2022] POST /identity/v3/users => generated 299 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:23:30.102078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c51d83-46b0-4eb0-8701-539557fb765e admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:30.113481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c51d83-46b0-4eb0-8701-539557fb765e admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:30.118402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450a1e0-a235-4e81-825b-b712d49ad805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:30.119037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450a1e0-a235-4e81-825b-b712d49ad805 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:30.119324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450a1e0-a235-4e81-825b-b712d49ad805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:30.119607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450a1e0-a235-4e81-825b-b712d49ad805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:30.120356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450a1e0-a235-4e81-825b-b712d49ad805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:30.122005 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450a1e0-a235-4e81-825b-b712d49ad805 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:30.133769 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1002/2025] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:30.138485 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b36d7-6884-454b-9090-ab9583518074 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:30.149483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b36d7-6884-454b-9090-ab9583518074 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:30.154085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e29331-ab93-41b2-afef-50bdd4fbc2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:30.154982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e29331-ab93-41b2-afef-50bdd4fbc2b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:30.155262 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e29331-ab93-41b2-afef-50bdd4fbc2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:30.155536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e29331-ab93-41b2-afef-50bdd4fbc2b4 admin admin] PATH_INFO: `/v3/projects/eadaf9521334496c9e0ee06b4a3b48ed/users/7ed5c4594c804d61822dcae32f8164a8/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:30.172016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e29331-ab93-41b2-afef-50bdd4fbc2b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eadaf9521334496c9e0ee06b4a3b48ed, user_id=7ed5c4594c804d61822dcae32f8164a8, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:30.176113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e29331-ab93-41b2-afef-50bdd4fbc2b4 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:30.303344 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1024/2026] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:23:30 2022] PUT /identity/v3/projects/eadaf9521334496c9e0ee06b4a3b48ed/users/7ed5c4594c804d61822dcae32f8164a8/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:30.307487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450a1e0-a235-4e81-825b-b712d49ad805 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:30.318503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450a1e0-a235-4e81-825b-b712d49ad805 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:30.339138 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:30.378578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf50ba-e52c-44a0-a201-d82698b88865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:30.379186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf50ba-e52c-44a0-a201-d82698b88865 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:30.379469 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf50ba-e52c-44a0-a201-d82698b88865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:30.379744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf50ba-e52c-44a0-a201-d82698b88865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:30.380571 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf50ba-e52c-44a0-a201-d82698b88865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:30.382595 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf50ba-e52c-44a0-a201-d82698b88865 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:30.393037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1003/2027] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:30.397668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e29331-ab93-41b2-afef-50bdd4fbc2b4 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:30.403333 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:30.409434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e29331-ab93-41b2-afef-50bdd4fbc2b4 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:30.414051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4ca06-32d8-4bfc-8423-3369a1b751e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:30.414952 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4ca06-32d8-4bfc-8423-3369a1b751e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:30.415276 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4ca06-32d8-4bfc-8423-3369a1b751e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:30.415512 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4ca06-32d8-4bfc-8423-3369a1b751e2 admin admin] PATH_INFO: `/v3/projects/eadaf9521334496c9e0ee06b4a3b48ed/users/7ed5c4594c804d61822dcae32f8164a8/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:30.417574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a4ca06-32d8-4bfc-8423-3369a1b751e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eadaf9521334496c9e0ee06b4a3b48ed, user_id=7ed5c4594c804d61822dcae32f8164a8, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:30.420241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a4ca06-32d8-4bfc-8423-3369a1b751e2 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:30.433645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1025/2028] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:23:30 2022] PUT /identity/v3/projects/eadaf9521334496c9e0ee06b4a3b48ed/users/7ed5c4594c804d61822dcae32f8164a8/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:30.452626 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807fa9a-a77d-46de-80e5-d7370bf159cb None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:30.453266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807fa9a-a77d-46de-80e5-d7370bf159cb None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:30.453561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807fa9a-a77d-46de-80e5-d7370bf159cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:30.483440 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-8807fa9a-a77d-46de-80e5-d7370bf159cb None None] MFA Rules not processed for user `7ed5c4594c804d61822dcae32f8164a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:30.484662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-8807fa9a-a77d-46de-80e5-d7370bf159cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:30.557855 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1004/2029] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:30.570093 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4ca06-32d8-4bfc-8423-3369a1b751e2 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:30.572703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-78a4ca06-32d8-4bfc-8423-3369a1b751e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:30.636054 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4ca06-32d8-4bfc-8423-3369a1b751e2 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:30.640311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42433e7-d831-4c43-9c38-318df7e986c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:30.640985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42433e7-d831-4c43-9c38-318df7e986c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:30.641284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42433e7-d831-4c43-9c38-318df7e986c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:30.641559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42433e7-d831-4c43-9c38-318df7e986c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:30.652022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42433e7-d831-4c43-9c38-318df7e986c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:30.653639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42433e7-d831-4c43-9c38-318df7e986c8 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:30.664394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1026/2030] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:31.816808 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807fa9a-a77d-46de-80e5-d7370bf159cb None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:31.881466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807fa9a-a77d-46de-80e5-d7370bf159cb None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:31.886081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db53637d-50d3-421e-b96d-347c5d3a1ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:31.886992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53637d-50d3-421e-b96d-347c5d3a1ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:31.887274 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53637d-50d3-421e-b96d-347c5d3a1ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:31.887552 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53637d-50d3-421e-b96d-347c5d3a1ec0 admin admin] PATH_INFO: `/v3/users/64e9366e183741a498eeb06834fcf99c` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:31.888745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53637d-50d3-421e-b96d-347c5d3a1ec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64e9366e183741a498eeb06834fcf99c)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:31.890694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53637d-50d3-421e-b96d-347c5d3a1ec0 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:32.117191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-db53637d-50d3-421e-b96d-347c5d3a1ec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64e9366e183741a498eeb06834fcf99c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:32.118739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:32.217109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-db53637d-50d3-421e-b96d-347c5d3a1ec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64e9366e183741a498eeb06834fcf99c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:32.217524 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-db53637d-50d3-421e-b96d-347c5d3a1ec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64e9366e183741a498eeb06834fcf99c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:32.218176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-db53637d-50d3-421e-b96d-347c5d3a1ec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64e9366e183741a498eeb06834fcf99c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:32.234740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-db53637d-50d3-421e-b96d-347c5d3a1ec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64e9366e183741a498eeb06834fcf99c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:32.249132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1005/2031] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:23:31 2022] DELETE /identity/v3/users/64e9366e183741a498eeb06834fcf99c => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:23:32.254495 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905c98b-d212-4eb0-ab80-c106c0709bfd None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:32.256008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905c98b-d212-4eb0-ab80-c106c0709bfd None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:32.256474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905c98b-d212-4eb0-ab80-c106c0709bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:32.275420 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-0905c98b-d212-4eb0-ab80-c106c0709bfd None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:32.276635 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0905c98b-d212-4eb0-ab80-c106c0709bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:32.346365 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1027/2032] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:32.356364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db53637d-50d3-421e-b96d-347c5d3a1ec0 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:32.362197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-db53637d-50d3-421e-b96d-347c5d3a1ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:32.421554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db53637d-50d3-421e-b96d-347c5d3a1ec0 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:32.425454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dd8ea-8f23-43c1-9e18-196670d2310b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:32.426112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dd8ea-8f23-43c1-9e18-196670d2310b service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:32.426395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dd8ea-8f23-43c1-9e18-196670d2310b service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:32.426671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dd8ea-8f23-43c1-9e18-196670d2310b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:32.435928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892dd8ea-8f23-43c1-9e18-196670d2310b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:32.437473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892dd8ea-8f23-43c1-9e18-196670d2310b service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:32.448554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1006/2033] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:32.880192 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905c98b-d212-4eb0-ab80-c106c0709bfd None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:32.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0905c98b-d212-4eb0-ab80-c106c0709bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:32.897234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905c98b-d212-4eb0-ab80-c106c0709bfd None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:32.902333 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374f6c6-f373-4408-a66f-ca416f20c1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:32.903345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374f6c6-f373-4408-a66f-ca416f20c1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:32.903587 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374f6c6-f373-4408-a66f-ca416f20c1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:32.904004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374f6c6-f373-4408-a66f-ca416f20c1d7 admin admin] PATH_INFO: `/v3/projects/32b28e20bdd54f09a7977fce5db00d2d` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:32.905699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374f6c6-f373-4408-a66f-ca416f20c1d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32b28e20bdd54f09a7977fce5db00d2d)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:32.907255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374f6c6-f373-4408-a66f-ca416f20c1d7 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:33.015587 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5374f6c6-f373-4408-a66f-ca416f20c1d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32b28e20bdd54f09a7977fce5db00d2d', 'request_id': 'req-5374f6c6-f373-4408-a66f-ca416f20c1d7'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:33.026449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5374f6c6-f373-4408-a66f-ca416f20c1d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32b28e20bdd54f09a7977fce5db00d2d', 'request_id': 'req-5374f6c6-f373-4408-a66f-ca416f20c1d7'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:33.032115 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5374f6c6-f373-4408-a66f-ca416f20c1d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32b28e20bdd54f09a7977fce5db00d2d', 'request_id': 'req-5374f6c6-f373-4408-a66f-ca416f20c1d7'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:33.107422 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5374f6c6-f373-4408-a66f-ca416f20c1d7 admin admin] The token cache is being invalidate because project 32b28e20bdd54f09a7977fce5db00d2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:23:33.107734 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5374f6c6-f373-4408-a66f-ca416f20c1d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:33.108355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5374f6c6-f373-4408-a66f-ca416f20c1d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:33.109963 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1028/2034] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:23:32 2022] DELETE /identity/v3/projects/32b28e20bdd54f09a7977fce5db00d2d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:23:33.114600 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dd8ea-8f23-43c1-9e18-196670d2310b service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:33.124259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dd8ea-8f23-43c1-9e18-196670d2310b service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:33.182457 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8c727-0436-49c7-a41c-2fbf20bd44d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:33.183326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8c727-0436-49c7-a41c-2fbf20bd44d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:33.183658 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8c727-0436-49c7-a41c-2fbf20bd44d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:33.183934 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8c727-0436-49c7-a41c-2fbf20bd44d5 admin admin] PATH_INFO: `/v3/users/7ed5c4594c804d61822dcae32f8164a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:33.185166 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8c727-0436-49c7-a41c-2fbf20bd44d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed5c4594c804d61822dcae32f8164a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:33.187078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8c727-0436-49c7-a41c-2fbf20bd44d5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:33.321758 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7db8c727-0436-49c7-a41c-2fbf20bd44d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed5c4594c804d61822dcae32f8164a8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:33.371023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7db8c727-0436-49c7-a41c-2fbf20bd44d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed5c4594c804d61822dcae32f8164a8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:33.371735 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7db8c727-0436-49c7-a41c-2fbf20bd44d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed5c4594c804d61822dcae32f8164a8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:33.372385 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7db8c727-0436-49c7-a41c-2fbf20bd44d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed5c4594c804d61822dcae32f8164a8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:33.386280 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:33.387788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7db8c727-0436-49c7-a41c-2fbf20bd44d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed5c4594c804d61822dcae32f8164a8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:33.400632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1007/2035] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:23:33 2022] DELETE /identity/v3/users/7ed5c4594c804d61822dcae32f8164a8 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:23:33.870103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374f6c6-f373-4408-a66f-ca416f20c1d7 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:33.871869 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5374f6c6-f373-4408-a66f-ca416f20c1d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:33.887531 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374f6c6-f373-4408-a66f-ca416f20c1d7 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:33.904477 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:33.951638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a040b-74ae-4e7c-835d-fcf56346bd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:33.952515 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a040b-74ae-4e7c-835d-fcf56346bd28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:33.952802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a040b-74ae-4e7c-835d-fcf56346bd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:33.953125 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a040b-74ae-4e7c-835d-fcf56346bd28 admin admin] PATH_INFO: `/v3/projects/eadaf9521334496c9e0ee06b4a3b48ed` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:33.954386 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a040b-74ae-4e7c-835d-fcf56346bd28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eadaf9521334496c9e0ee06b4a3b48ed)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:33.957139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a040b-74ae-4e7c-835d-fcf56346bd28 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:34.097523 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6e5a040b-74ae-4e7c-835d-fcf56346bd28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eadaf9521334496c9e0ee06b4a3b48ed', 'request_id': 'req-6e5a040b-74ae-4e7c-835d-fcf56346bd28'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:34.103845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6e5a040b-74ae-4e7c-835d-fcf56346bd28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eadaf9521334496c9e0ee06b4a3b48ed', 'request_id': 'req-6e5a040b-74ae-4e7c-835d-fcf56346bd28'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:34.109868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6e5a040b-74ae-4e7c-835d-fcf56346bd28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eadaf9521334496c9e0ee06b4a3b48ed', 'request_id': 'req-6e5a040b-74ae-4e7c-835d-fcf56346bd28'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:34.188058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6e5a040b-74ae-4e7c-835d-fcf56346bd28 admin admin] The token cache is being invalidate because project eadaf9521334496c9e0ee06b4a3b48ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:23:34.188350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6e5a040b-74ae-4e7c-835d-fcf56346bd28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:34.189005 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6e5a040b-74ae-4e7c-835d-fcf56346bd28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:34.191065 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1029/2036] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:23:33 2022] DELETE /identity/v3/projects/eadaf9521334496c9e0ee06b4a3b48ed => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:34.196483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f3ceb-ec16-49b0-8c85-f22192f3c393 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:34.197179 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f3ceb-ec16-49b0-8c85-f22192f3c393 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:34.197179 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f3ceb-ec16-49b0-8c85-f22192f3c393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:34.215995 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-851f3ceb-ec16-49b0-8c85-f22192f3c393 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:34.217257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-851f3ceb-ec16-49b0-8c85-f22192f3c393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:34.278427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1008/2037] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:34.286850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92198cd9-319b-49f9-93f0-4ce6de7e274e None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:34.287144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92198cd9-319b-49f9-93f0-4ce6de7e274e None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:34.287661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92198cd9-319b-49f9-93f0-4ce6de7e274e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:34.306987 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-92198cd9-319b-49f9-93f0-4ce6de7e274e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:34.309085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-92198cd9-319b-49f9-93f0-4ce6de7e274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:34.319052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1030/2038] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:23:34.323355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f3ceb-ec16-49b0-8c85-f22192f3c393 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:34.333550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f3ceb-ec16-49b0-8c85-f22192f3c393 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:34.338428 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcde28-6f0c-48c9-9589-1d0665b093f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:34.339090 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcde28-6f0c-48c9-9589-1d0665b093f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:34.339380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcde28-6f0c-48c9-9589-1d0665b093f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:34.339672 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fcde28-6f0c-48c9-9589-1d0665b093f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:34.340415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fcde28-6f0c-48c9-9589-1d0665b093f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:23:34.340777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fcde28-6f0c-48c9-9589-1d0665b093f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:34.342460 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fcde28-6f0c-48c9-9589-1d0665b093f1 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:34.354931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1009/2039] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:23:34.359207 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92198cd9-319b-49f9-93f0-4ce6de7e274e None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:34.369385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92198cd9-319b-49f9-93f0-4ce6de7e274e None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:34.374723 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195dbe7-f9e6-46f2-96b0-a85b7e8aa442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:34.375521 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195dbe7-f9e6-46f2-96b0-a85b7e8aa442 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:34.376017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195dbe7-f9e6-46f2-96b0-a85b7e8aa442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:34.376478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c195dbe7-f9e6-46f2-96b0-a85b7e8aa442 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:34.378615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c195dbe7-f9e6-46f2-96b0-a85b7e8aa442 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:34.380638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c195dbe7-f9e6-46f2-96b0-a85b7e8aa442 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:34.519827 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1031/2040] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:23:34 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:23:34.524768 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcde28-6f0c-48c9-9589-1d0665b093f1 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:34.535531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fcde28-6f0c-48c9-9589-1d0665b093f1 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:34.596101 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca28b0-a959-4a41-8590-5896f4cdd82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:34.596745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca28b0-a959-4a41-8590-5896f4cdd82c admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:34.597057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca28b0-a959-4a41-8590-5896f4cdd82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:34.597332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca28b0-a959-4a41-8590-5896f4cdd82c admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:34.598241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca28b0-a959-4a41-8590-5896f4cdd82c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:34.600178 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca28b0-a959-4a41-8590-5896f4cdd82c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:34.701929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1010/2041] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:23:34 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:23:34.703297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195dbe7-f9e6-46f2-96b0-a85b7e8aa442 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:34.715444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c195dbe7-f9e6-46f2-96b0-a85b7e8aa442 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:34.721808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b73012-c8f2-49f5-9191-8a8011482eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:34.722762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b73012-c8f2-49f5-9191-8a8011482eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:34.723046 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b73012-c8f2-49f5-9191-8a8011482eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:34.723324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b73012-c8f2-49f5-9191-8a8011482eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:34.724279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b73012-c8f2-49f5-9191-8a8011482eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:34.725950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b73012-c8f2-49f5-9191-8a8011482eb7 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:34.741033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1032/2042] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:34.744066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca28b0-a959-4a41-8590-5896f4cdd82c admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:34.759846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca28b0-a959-4a41-8590-5896f4cdd82c admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:34.769534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b625df0-93cc-44d1-b9b5-83cd60f981c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:34.769534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b625df0-93cc-44d1-b9b5-83cd60f981c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:34.769534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b625df0-93cc-44d1-b9b5-83cd60f981c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:34.769534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b625df0-93cc-44d1-b9b5-83cd60f981c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:34.769869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b625df0-93cc-44d1-b9b5-83cd60f981c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:34.769869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b625df0-93cc-44d1-b9b5-83cd60f981c0 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:34.780531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1011/2043] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:34.785773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b73012-c8f2-49f5-9191-8a8011482eb7 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:34.800832 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b73012-c8f2-49f5-9191-8a8011482eb7 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:34.808310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20746886-fa0c-4e81-b4a6-4eae42699a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:34.809859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20746886-fa0c-4e81-b4a6-4eae42699a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:34.810333 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20746886-fa0c-4e81-b4a6-4eae42699a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:34.810819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20746886-fa0c-4e81-b4a6-4eae42699a70 admin admin] PATH_INFO: `/v3/projects/67365011957d4e339f027a92c3f159f7/users/4618a3f4c7954d12b6c63fde79602b2d/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:34.836625 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20746886-fa0c-4e81-b4a6-4eae42699a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67365011957d4e339f027a92c3f159f7, user_id=4618a3f4c7954d12b6c63fde79602b2d, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:34.839379 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20746886-fa0c-4e81-b4a6-4eae42699a70 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:34.936014 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1033/2044] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:23:34 2022] PUT /identity/v3/projects/67365011957d4e339f027a92c3f159f7/users/4618a3f4c7954d12b6c63fde79602b2d/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:34.947948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b535abc-b511-4b0a-91b2-8a84c9e0d326 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:34.948261 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b535abc-b511-4b0a-91b2-8a84c9e0d326 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:34.948543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b535abc-b511-4b0a-91b2-8a84c9e0d326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:34.978944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-8b535abc-b511-4b0a-91b2-8a84c9e0d326 None None] MFA Rules not processed for user `4618a3f4c7954d12b6c63fde79602b2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:34.980239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-8b535abc-b511-4b0a-91b2-8a84c9e0d326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:35.025551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:35.051336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1012/2045] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:35.207313 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20746886-fa0c-4e81-b4a6-4eae42699a70 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:35.209485 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-20746886-fa0c-4e81-b4a6-4eae42699a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:35.258132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:35.270652 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20746886-fa0c-4e81-b4a6-4eae42699a70 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:35.274281 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de65090-9c88-47ca-bf04-988bed8513f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:35.274927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de65090-9c88-47ca-bf04-988bed8513f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:35.275206 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de65090-9c88-47ca-bf04-988bed8513f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:35.275481 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de65090-9c88-47ca-bf04-988bed8513f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:35.284780 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de65090-9c88-47ca-bf04-988bed8513f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:35.286242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de65090-9c88-47ca-bf04-988bed8513f7 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:35.295721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1034/2046] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:35.322412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b535abc-b511-4b0a-91b2-8a84c9e0d326 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:35.397696 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b535abc-b511-4b0a-91b2-8a84c9e0d326 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:35.402492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7453b-d5ae-44b3-874e-34c35f6803f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:35.404158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7453b-d5ae-44b3-874e-34c35f6803f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:35.404158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7453b-d5ae-44b3-874e-34c35f6803f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:35.404158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7453b-d5ae-44b3-874e-34c35f6803f7 admin admin] PATH_INFO: `/v3/users/4618a3f4c7954d12b6c63fde79602b2d` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:35.405320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7453b-d5ae-44b3-874e-34c35f6803f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4618a3f4c7954d12b6c63fde79602b2d)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:35.407228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7453b-d5ae-44b3-874e-34c35f6803f7 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:35.632755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-70c7453b-d5ae-44b3-874e-34c35f6803f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4618a3f4c7954d12b6c63fde79602b2d'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:35.685516 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-70c7453b-d5ae-44b3-874e-34c35f6803f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4618a3f4c7954d12b6c63fde79602b2d'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:35.686153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-70c7453b-d5ae-44b3-874e-34c35f6803f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4618a3f4c7954d12b6c63fde79602b2d'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:35.686766 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-70c7453b-d5ae-44b3-874e-34c35f6803f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4618a3f4c7954d12b6c63fde79602b2d'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:35.701192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-70c7453b-d5ae-44b3-874e-34c35f6803f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4618a3f4c7954d12b6c63fde79602b2d'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:35.730912 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1013/2047] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:23:35 2022] DELETE /identity/v3/users/4618a3f4c7954d12b6c63fde79602b2d => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:35.735468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3040f9-ff01-4ab0-91ff-9af84e586c2f None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:35.735756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3040f9-ff01-4ab0-91ff-9af84e586c2f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:35.736802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3040f9-ff01-4ab0-91ff-9af84e586c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:35.756424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-af3040f9-ff01-4ab0-91ff-9af84e586c2f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:35.757708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-af3040f9-ff01-4ab0-91ff-9af84e586c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:35.831874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1035/2048] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:35.848440 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7453b-d5ae-44b3-874e-34c35f6803f7 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:35.850277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-70c7453b-d5ae-44b3-874e-34c35f6803f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:35.921590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7453b-d5ae-44b3-874e-34c35f6803f7 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:35.926157 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d117a-5cbc-4ee1-9cc9-1b1b18fe818a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:35.926844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d117a-5cbc-4ee1-9cc9-1b1b18fe818a service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:35.927132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d117a-5cbc-4ee1-9cc9-1b1b18fe818a service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:35.927407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d117a-5cbc-4ee1-9cc9-1b1b18fe818a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:35.937706 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d117a-5cbc-4ee1-9cc9-1b1b18fe818a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:35.939171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d117a-5cbc-4ee1-9cc9-1b1b18fe818a service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:35.950938 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1014/2049] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:36.384152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3040f9-ff01-4ab0-91ff-9af84e586c2f None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:36.385658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-af3040f9-ff01-4ab0-91ff-9af84e586c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:36.398680 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3040f9-ff01-4ab0-91ff-9af84e586c2f None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:36.403567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef01755-7419-4fd8-8edd-f2c90fdf6c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:36.404381 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01755-7419-4fd8-8edd-f2c90fdf6c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:36.404662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01755-7419-4fd8-8edd-f2c90fdf6c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:36.404933 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01755-7419-4fd8-8edd-f2c90fdf6c66 admin admin] PATH_INFO: `/v3/projects/67365011957d4e339f027a92c3f159f7` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:36.406218 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef01755-7419-4fd8-8edd-f2c90fdf6c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67365011957d4e339f027a92c3f159f7)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:36.409509 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef01755-7419-4fd8-8edd-f2c90fdf6c66 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:36.513056 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cef01755-7419-4fd8-8edd-f2c90fdf6c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67365011957d4e339f027a92c3f159f7', 'request_id': 'req-cef01755-7419-4fd8-8edd-f2c90fdf6c66'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:36.520710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cef01755-7419-4fd8-8edd-f2c90fdf6c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67365011957d4e339f027a92c3f159f7', 'request_id': 'req-cef01755-7419-4fd8-8edd-f2c90fdf6c66'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:36.527755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cef01755-7419-4fd8-8edd-f2c90fdf6c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67365011957d4e339f027a92c3f159f7', 'request_id': 'req-cef01755-7419-4fd8-8edd-f2c90fdf6c66'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:36.608766 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cef01755-7419-4fd8-8edd-f2c90fdf6c66 admin admin] The token cache is being invalidate because project 67365011957d4e339f027a92c3f159f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:23:36.609347 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cef01755-7419-4fd8-8edd-f2c90fdf6c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:36.609980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-cef01755-7419-4fd8-8edd-f2c90fdf6c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:36.611861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1036/2050] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:23:36 2022] DELETE /identity/v3/projects/67365011957d4e339f027a92c3f159f7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:23:36.616822 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b980c-268f-4cc3-bb1d-e36419e3255e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:36.617339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b980c-268f-4cc3-bb1d-e36419e3255e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:36.617430 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b980c-268f-4cc3-bb1d-e36419e3255e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:36.624933 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:36.638262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-e37b980c-268f-4cc3-bb1d-e36419e3255e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:36.639399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e37b980c-268f-4cc3-bb1d-e36419e3255e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:36.700601 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1015/2051] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:36.709817 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17d2d3-864c-4745-8bbf-8f580f337417 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:36.710105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17d2d3-864c-4745-8bbf-8f580f337417 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:36.710387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17d2d3-864c-4745-8bbf-8f580f337417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:36.729708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-cb17d2d3-864c-4745-8bbf-8f580f337417 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:36.730959 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-cb17d2d3-864c-4745-8bbf-8f580f337417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:36.739733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1037/2052] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:36.744060 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b980c-268f-4cc3-bb1d-e36419e3255e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:36.755102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b980c-268f-4cc3-bb1d-e36419e3255e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:36.760745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee4385-adb7-49c0-806c-7ca9b0eb5808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:36.761454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee4385-adb7-49c0-806c-7ca9b0eb5808 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:36.761744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee4385-adb7-49c0-806c-7ca9b0eb5808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:36.762026 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee4385-adb7-49c0-806c-7ca9b0eb5808 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:36.762786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee4385-adb7-49c0-806c-7ca9b0eb5808 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:23:36.763087 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee4385-adb7-49c0-806c-7ca9b0eb5808 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:36.764823 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee4385-adb7-49c0-806c-7ca9b0eb5808 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:36.781240 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1016/2053] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:23:36.782404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17d2d3-864c-4745-8bbf-8f580f337417 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:36.793084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17d2d3-864c-4745-8bbf-8f580f337417 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:36.797817 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8b3b9-3eaa-42dc-9544-ef7b31d6e626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:36.798560 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8b3b9-3eaa-42dc-9544-ef7b31d6e626 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:36.799054 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8b3b9-3eaa-42dc-9544-ef7b31d6e626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:36.799333 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8b3b9-3eaa-42dc-9544-ef7b31d6e626 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:36.800052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8b3b9-3eaa-42dc-9544-ef7b31d6e626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:36.801994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8b3b9-3eaa-42dc-9544-ef7b31d6e626 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:36.881414 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1038/2054] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:23:36 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:23:36.885750 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee4385-adb7-49c0-806c-7ca9b0eb5808 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:36.896878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee4385-adb7-49c0-806c-7ca9b0eb5808 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:36.959972 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b0f8e-0bf3-45d7-85f7-27e17918a1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:36.960686 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b0f8e-0bf3-45d7-85f7-27e17918a1a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:36.961099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b0f8e-0bf3-45d7-85f7-27e17918a1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:36.961276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b0f8e-0bf3-45d7-85f7-27e17918a1a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:36.962066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b0f8e-0bf3-45d7-85f7-27e17918a1a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:36.964153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b0f8e-0bf3-45d7-85f7-27e17918a1a3 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:37.092420 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1017/2055] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:23:36 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:23:37.098680 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8b3b9-3eaa-42dc-9544-ef7b31d6e626 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:37.109324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8b3b9-3eaa-42dc-9544-ef7b31d6e626 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:37.121164 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158c48f-d26b-40fe-bbe9-73564799edd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:37.121845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158c48f-d26b-40fe-bbe9-73564799edd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:37.122129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158c48f-d26b-40fe-bbe9-73564799edd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:37.122405 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158c48f-d26b-40fe-bbe9-73564799edd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:37.123141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158c48f-d26b-40fe-bbe9-73564799edd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:37.124745 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158c48f-d26b-40fe-bbe9-73564799edd1 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:37.135919 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1039/2056] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:37.140177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b0f8e-0bf3-45d7-85f7-27e17918a1a3 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:37.150778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b0f8e-0bf3-45d7-85f7-27e17918a1a3 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:37.155405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21e782-ffa1-4b61-81d5-d1bcb7d1a39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:37.156071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21e782-ffa1-4b61-81d5-d1bcb7d1a39d admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:37.156360 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21e782-ffa1-4b61-81d5-d1bcb7d1a39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:37.156645 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21e782-ffa1-4b61-81d5-d1bcb7d1a39d admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:37.157426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21e782-ffa1-4b61-81d5-d1bcb7d1a39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:37.159761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21e782-ffa1-4b61-81d5-d1bcb7d1a39d admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:37.170205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1018/2057] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:37.175151 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158c48f-d26b-40fe-bbe9-73564799edd1 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:37.185201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158c48f-d26b-40fe-bbe9-73564799edd1 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:37.190633 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2abb9-d666-4d85-81cf-2e11cf232e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:37.191527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2abb9-d666-4d85-81cf-2e11cf232e1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:37.191810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2abb9-d666-4d85-81cf-2e11cf232e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:37.192090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2abb9-d666-4d85-81cf-2e11cf232e1e admin admin] PATH_INFO: `/v3/projects/2cf7398c2e3846eba51beeaacddfb685/users/2de05736c1234a34a7792621309a0658/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:37.209116 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2abb9-d666-4d85-81cf-2e11cf232e1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cf7398c2e3846eba51beeaacddfb685, user_id=2de05736c1234a34a7792621309a0658, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:37.211800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2abb9-d666-4d85-81cf-2e11cf232e1e admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:37.399771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1040/2058] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:23:37 2022] PUT /identity/v3/projects/2cf7398c2e3846eba51beeaacddfb685/users/2de05736c1234a34a7792621309a0658/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:37.411831 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7248aa-fc85-499a-bcd0-35be2e79ec73 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:37.412061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7248aa-fc85-499a-bcd0-35be2e79ec73 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:37.412331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7248aa-fc85-499a-bcd0-35be2e79ec73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:37.441342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-1f7248aa-fc85-499a-bcd0-35be2e79ec73 None None] MFA Rules not processed for user `2de05736c1234a34a7792621309a0658`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:37.442588 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1f7248aa-fc85-499a-bcd0-35be2e79ec73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:37.505925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:37.509960 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1019/2059] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:37.521381 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2abb9-d666-4d85-81cf-2e11cf232e1e admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:37.523139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c2e2abb9-d666-4d85-81cf-2e11cf232e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:37.529822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:37.582152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2abb9-d666-4d85-81cf-2e11cf232e1e admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:37.585930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c1f1c-f045-4c26-8217-eea088ff135f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fbf33189aa94f4f973d558d615a23e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:37.586668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c1f1c-f045-4c26-8217-eea088ff135f service glance] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:37.586934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c1f1c-f045-4c26-8217-eea088ff135f service glance] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:37.587214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c1f1c-f045-4c26-8217-eea088ff135f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:37.596733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c1f1c-f045-4c26-8217-eea088ff135f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:37.598246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c1f1c-f045-4c26-8217-eea088ff135f service glance] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:37.607242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1041/2060] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:47.719136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=24 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:48.133301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7248aa-fc85-499a-bcd0-35be2e79ec73 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:48.134835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1f7248aa-fc85-499a-bcd0-35be2e79ec73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:48.249221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7248aa-fc85-499a-bcd0-35be2e79ec73 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:48.254314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba59404-0129-4d13-8c37-0623ed8b7282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:48.255231 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba59404-0129-4d13-8c37-0623ed8b7282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:48.255520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba59404-0129-4d13-8c37-0623ed8b7282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:48.255797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba59404-0129-4d13-8c37-0623ed8b7282 admin admin] PATH_INFO: `/v3/users/179d85a2dc7744b79b7111e4a0a20256` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:48.257080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba59404-0129-4d13-8c37-0623ed8b7282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=179d85a2dc7744b79b7111e4a0a20256)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:48.258990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba59404-0129-4d13-8c37-0623ed8b7282 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:48.435492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-eba59404-0129-4d13-8c37-0623ed8b7282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '179d85a2dc7744b79b7111e4a0a20256'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:48.495122 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-eba59404-0129-4d13-8c37-0623ed8b7282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '179d85a2dc7744b79b7111e4a0a20256'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:48.495835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-eba59404-0129-4d13-8c37-0623ed8b7282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '179d85a2dc7744b79b7111e4a0a20256'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:48.496468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-eba59404-0129-4d13-8c37-0623ed8b7282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '179d85a2dc7744b79b7111e4a0a20256'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:48.510812 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-eba59404-0129-4d13-8c37-0623ed8b7282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '179d85a2dc7744b79b7111e4a0a20256'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:48.524042 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1020/2061] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:23:48 2022] DELETE /identity/v3/users/179d85a2dc7744b79b7111e4a0a20256 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:48.529360 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87798442-7975-48f1-ac3e-4a1eb2aa90b7 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:48.529654 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87798442-7975-48f1-ac3e-4a1eb2aa90b7 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:48.529937 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87798442-7975-48f1-ac3e-4a1eb2aa90b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:48.549825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-87798442-7975-48f1-ac3e-4a1eb2aa90b7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:48.551094 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-87798442-7975-48f1-ac3e-4a1eb2aa90b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:48.640926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1042/2062] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:48.653084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba59404-0129-4d13-8c37-0623ed8b7282 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:48.656610 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-eba59404-0129-4d13-8c37-0623ed8b7282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:48.670060 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cda4a7-7434-4874-bcd0-1f29d1e7c97a None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:48.670378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cda4a7-7434-4874-bcd0-1f29d1e7c97a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:48.670808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cda4a7-7434-4874-bcd0-1f29d1e7c97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:48.693831 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-49cda4a7-7434-4874-bcd0-1f29d1e7c97a None None] MFA Rules not processed for user `1d4be529ee5e4765ada8eaee33e21c64`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:48.695433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-49cda4a7-7434-4874-bcd0-1f29d1e7c97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:48.708815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:48.767139 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba59404-0129-4d13-8c37-0623ed8b7282 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:48.767324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1043/2063] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:48.772738 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19adc1-7116-43c5-a719-3edcbc7e4906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:48.773459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19adc1-7116-43c5-a719-3edcbc7e4906 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:48.773750 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19adc1-7116-43c5-a719-3edcbc7e4906 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:48.774036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19adc1-7116-43c5-a719-3edcbc7e4906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:48.786118 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19adc1-7116-43c5-a719-3edcbc7e4906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:48.787640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19adc1-7116-43c5-a719-3edcbc7e4906 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:48.805228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1021/2064] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:23:49.418929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cda4a7-7434-4874-bcd0-1f29d1e7c97a None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:49.420384 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-49cda4a7-7434-4874-bcd0-1f29d1e7c97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:49.435121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cda4a7-7434-4874-bcd0-1f29d1e7c97a None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:49.441040 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac652f-8138-4f73-9142-60021a65c550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:49.441820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac652f-8138-4f73-9142-60021a65c550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:49.442089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac652f-8138-4f73-9142-60021a65c550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:49.442372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac652f-8138-4f73-9142-60021a65c550 admin admin] PATH_INFO: `/v3/projects/cb9ef70beebf460ca603c258705bfa69` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:49.443595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac652f-8138-4f73-9142-60021a65c550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb9ef70beebf460ca603c258705bfa69)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:49.445908 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac652f-8138-4f73-9142-60021a65c550 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:49.608382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-86ac652f-8138-4f73-9142-60021a65c550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb9ef70beebf460ca603c258705bfa69', 'request_id': 'req-86ac652f-8138-4f73-9142-60021a65c550'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:49.617009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-86ac652f-8138-4f73-9142-60021a65c550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb9ef70beebf460ca603c258705bfa69', 'request_id': 'req-86ac652f-8138-4f73-9142-60021a65c550'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:49.635282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-86ac652f-8138-4f73-9142-60021a65c550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb9ef70beebf460ca603c258705bfa69', 'request_id': 'req-86ac652f-8138-4f73-9142-60021a65c550'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:49.714193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-86ac652f-8138-4f73-9142-60021a65c550 admin admin] The token cache is being invalidate because project cb9ef70beebf460ca603c258705bfa69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:23:49.714482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-86ac652f-8138-4f73-9142-60021a65c550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:49.715234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-86ac652f-8138-4f73-9142-60021a65c550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:23:49.716787 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1044/2065] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:23:49 2022] DELETE /identity/v3/projects/cb9ef70beebf460ca603c258705bfa69 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:49.722701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67670dc-80be-473b-93d5-7d294f6dd21e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:49.723253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67670dc-80be-473b-93d5-7d294f6dd21e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:49.723386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67670dc-80be-473b-93d5-7d294f6dd21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:49.771469 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b67670dc-80be-473b-93d5-7d294f6dd21e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:49.772715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b67670dc-80be-473b-93d5-7d294f6dd21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:49.855855 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1022/2066] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:49.864413 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac43b77-8537-4512-b8eb-4d5ceb84c3f6 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:49.864911 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac43b77-8537-4512-b8eb-4d5ceb84c3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:49.867334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac43b77-8537-4512-b8eb-4d5ceb84c3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:49.884867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-aac43b77-8537-4512-b8eb-4d5ceb84c3f6 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:49.886145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-aac43b77-8537-4512-b8eb-4d5ceb84c3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:49.895335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1045/2067] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:23:49.899458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67670dc-80be-473b-93d5-7d294f6dd21e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:49.911033 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67670dc-80be-473b-93d5-7d294f6dd21e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:49.915858 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f111476-6658-4665-9db3-88f481fcaec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:49.916527 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f111476-6658-4665-9db3-88f481fcaec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:49.916816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f111476-6658-4665-9db3-88f481fcaec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:49.917126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f111476-6658-4665-9db3-88f481fcaec3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:49.917923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f111476-6658-4665-9db3-88f481fcaec3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:23:49.918202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f111476-6658-4665-9db3-88f481fcaec3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:49.919802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f111476-6658-4665-9db3-88f481fcaec3 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:49.932918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1023/2068] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:23:49.937462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac43b77-8537-4512-b8eb-4d5ceb84c3f6 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:49.950140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac43b77-8537-4512-b8eb-4d5ceb84c3f6 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:49.955425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367e434-e6bc-4f6e-a8a2-8dfc4388221a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:49.956132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367e434-e6bc-4f6e-a8a2-8dfc4388221a admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:49.956446 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367e434-e6bc-4f6e-a8a2-8dfc4388221a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:49.956726 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2367e434-e6bc-4f6e-a8a2-8dfc4388221a admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:49.957479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367e434-e6bc-4f6e-a8a2-8dfc4388221a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:49.959399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2367e434-e6bc-4f6e-a8a2-8dfc4388221a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:50.149618 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:50.156085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1046/2069] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:23:49 2022] POST /identity/v3/projects => generated 399 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:23:50.162215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f111476-6658-4665-9db3-88f481fcaec3 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:50.172474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f111476-6658-4665-9db3-88f481fcaec3 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:50.224281 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:23:50.234251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29a427-7a05-474b-952c-f29fa3ab6ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:50.234944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29a427-7a05-474b-952c-f29fa3ab6ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:50.235235 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29a427-7a05-474b-952c-f29fa3ab6ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:50.235515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29a427-7a05-474b-952c-f29fa3ab6ef0 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:50.236309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29a427-7a05-474b-952c-f29fa3ab6ef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:50.238337 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29a427-7a05-474b-952c-f29fa3ab6ef0 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:50.347323 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1024/2070] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:23:50 2022] POST /identity/v3/users => generated 300 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:23:50.351575 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367e434-e6bc-4f6e-a8a2-8dfc4388221a admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:50.362365 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2367e434-e6bc-4f6e-a8a2-8dfc4388221a admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:50.367120 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2ebfb-d7fe-49d6-b469-b0772d07cca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:50.367879 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2ebfb-d7fe-49d6-b469-b0772d07cca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:50.368162 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2ebfb-d7fe-49d6-b469-b0772d07cca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:50.368436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2ebfb-d7fe-49d6-b469-b0772d07cca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:50.369205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2ebfb-d7fe-49d6-b469-b0772d07cca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:50.370806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2ebfb-d7fe-49d6-b469-b0772d07cca9 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:50.385202 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1047/2071] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:50.386378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29a427-7a05-474b-952c-f29fa3ab6ef0 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:50.397170 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29a427-7a05-474b-952c-f29fa3ab6ef0 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:50.401745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17423c35-e0b3-431f-8ace-fdbf6963b37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:50.402417 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17423c35-e0b3-431f-8ace-fdbf6963b37d admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:50.402700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17423c35-e0b3-431f-8ace-fdbf6963b37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:50.402978 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17423c35-e0b3-431f-8ace-fdbf6963b37d admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:50.403714 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17423c35-e0b3-431f-8ace-fdbf6963b37d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:50.405345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17423c35-e0b3-431f-8ace-fdbf6963b37d admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:50.415558 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1025/2072] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:23:50.420139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2ebfb-d7fe-49d6-b469-b0772d07cca9 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:50.430363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2ebfb-d7fe-49d6-b469-b0772d07cca9 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:50.435028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d8f6d-2003-439f-9649-c3ba7846df96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:50.436559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d8f6d-2003-439f-9649-c3ba7846df96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:50.436559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d8f6d-2003-439f-9649-c3ba7846df96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:50.436559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d8f6d-2003-439f-9649-c3ba7846df96 admin admin] PATH_INFO: `/v3/projects/045b3dd97b0c42f1b639ca09ba6adbc4/users/10e73732fa1743a18a72b74654d60c74/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:50.452547 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d8f6d-2003-439f-9649-c3ba7846df96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=045b3dd97b0c42f1b639ca09ba6adbc4, user_id=10e73732fa1743a18a72b74654d60c74, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:50.456074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d8f6d-2003-439f-9649-c3ba7846df96 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:50.558955 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1048/2073] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:23:50 2022] PUT /identity/v3/projects/045b3dd97b0c42f1b639ca09ba6adbc4/users/10e73732fa1743a18a72b74654d60c74/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:23:50.570819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57108aaa-4f26-4baa-962e-ee7b834794a6 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:50.571111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57108aaa-4f26-4baa-962e-ee7b834794a6 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:50.571394 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57108aaa-4f26-4baa-962e-ee7b834794a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:50.599391 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-57108aaa-4f26-4baa-962e-ee7b834794a6 None None] MFA Rules not processed for user `10e73732fa1743a18a72b74654d60c74`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:23:50.600606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-57108aaa-4f26-4baa-962e-ee7b834794a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:50.669007 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1026/2074] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:23:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:23:50.680926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d8f6d-2003-439f-9649-c3ba7846df96 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:23:50.682905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-a31d8f6d-2003-439f-9649-c3ba7846df96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:23:50.759373 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d8f6d-2003-439f-9649-c3ba7846df96 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:23:50.763043 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f79dab-ae66-4c9a-84a1-0a8dea2858ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:23:50.763685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f79dab-ae66-4c9a-84a1-0a8dea2858ec service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:23:50.763967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f79dab-ae66-4c9a-84a1-0a8dea2858ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:23:50.764241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f79dab-ae66-4c9a-84a1-0a8dea2858ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:23:50.772705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f79dab-ae66-4c9a-84a1-0a8dea2858ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:23:50.774187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f79dab-ae66-4c9a-84a1-0a8dea2858ec service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:23:50.783308 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1049/2075] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:23:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:00.665766 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=43,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:00.779139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:13.483196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57108aaa-4f26-4baa-962e-ee7b834794a6 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:13.549845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57108aaa-4f26-4baa-962e-ee7b834794a6 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:13.554642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3f19b-2708-40fb-9743-b57b6c5a8d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:13.555520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3f19b-2708-40fb-9743-b57b6c5a8d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:13.555803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3f19b-2708-40fb-9743-b57b6c5a8d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:13.556081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3f19b-2708-40fb-9743-b57b6c5a8d1d admin admin] PATH_INFO: `/v3/users/10e73732fa1743a18a72b74654d60c74` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:13.557321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3f19b-2708-40fb-9743-b57b6c5a8d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=10e73732fa1743a18a72b74654d60c74)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:13.559965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3f19b-2708-40fb-9743-b57b6c5a8d1d admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:13.781279 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-07d3f19b-2708-40fb-9743-b57b6c5a8d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10e73732fa1743a18a72b74654d60c74'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:13.880441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-07d3f19b-2708-40fb-9743-b57b6c5a8d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10e73732fa1743a18a72b74654d60c74'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:13.881122 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-07d3f19b-2708-40fb-9743-b57b6c5a8d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10e73732fa1743a18a72b74654d60c74'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:13.881749 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-07d3f19b-2708-40fb-9743-b57b6c5a8d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10e73732fa1743a18a72b74654d60c74'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:13.895611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-07d3f19b-2708-40fb-9743-b57b6c5a8d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10e73732fa1743a18a72b74654d60c74'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:13.910388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1027/2076] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:24:13 2022] DELETE /identity/v3/users/10e73732fa1743a18a72b74654d60c74 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:13.915142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd768a67-8ee7-48b2-9fd1-bb60b6efd861 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:13.915653 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd768a67-8ee7-48b2-9fd1-bb60b6efd861 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:13.915831 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd768a67-8ee7-48b2-9fd1-bb60b6efd861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:13.937974 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-dd768a67-8ee7-48b2-9fd1-bb60b6efd861 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:13.937974 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-dd768a67-8ee7-48b2-9fd1-bb60b6efd861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:13.997600 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1050/2077] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:24:14.007406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3f19b-2708-40fb-9743-b57b6c5a8d1d admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:14.009059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-07d3f19b-2708-40fb-9743-b57b6c5a8d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:14.071282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3f19b-2708-40fb-9743-b57b6c5a8d1d admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:14.078589 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f3fd7-f448-41d0-a9e2-42c063d3a8c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:14.079250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f3fd7-f448-41d0-a9e2-42c063d3a8c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:14.079531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f3fd7-f448-41d0-a9e2-42c063d3a8c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:14.079858 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f3fd7-f448-41d0-a9e2-42c063d3a8c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:14.089079 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f3fd7-f448-41d0-a9e2-42c063d3a8c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:14.091622 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f3fd7-f448-41d0-a9e2-42c063d3a8c3 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:14.101039 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1028/2078] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:14.505791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd768a67-8ee7-48b2-9fd1-bb60b6efd861 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:14.507321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-dd768a67-8ee7-48b2-9fd1-bb60b6efd861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:14.520348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd768a67-8ee7-48b2-9fd1-bb60b6efd861 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:14.528147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5493cac-2ab5-4428-aa3b-18735d8d5e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:14.529070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5493cac-2ab5-4428-aa3b-18735d8d5e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:14.531257 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5493cac-2ab5-4428-aa3b-18735d8d5e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:14.531257 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5493cac-2ab5-4428-aa3b-18735d8d5e4d admin admin] PATH_INFO: `/v3/projects/045b3dd97b0c42f1b639ca09ba6adbc4` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:14.531257 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5493cac-2ab5-4428-aa3b-18735d8d5e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=045b3dd97b0c42f1b639ca09ba6adbc4)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:14.533084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5493cac-2ab5-4428-aa3b-18735d8d5e4d admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:14.652687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f5493cac-2ab5-4428-aa3b-18735d8d5e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '045b3dd97b0c42f1b639ca09ba6adbc4', 'request_id': 'req-f5493cac-2ab5-4428-aa3b-18735d8d5e4d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:14.659683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f5493cac-2ab5-4428-aa3b-18735d8d5e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '045b3dd97b0c42f1b639ca09ba6adbc4', 'request_id': 'req-f5493cac-2ab5-4428-aa3b-18735d8d5e4d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:14.665481 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f5493cac-2ab5-4428-aa3b-18735d8d5e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '045b3dd97b0c42f1b639ca09ba6adbc4', 'request_id': 'req-f5493cac-2ab5-4428-aa3b-18735d8d5e4d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:14.742524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f5493cac-2ab5-4428-aa3b-18735d8d5e4d admin admin] The token cache is being invalidate because project 045b3dd97b0c42f1b639ca09ba6adbc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:24:14.742806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f5493cac-2ab5-4428-aa3b-18735d8d5e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:14.743417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f5493cac-2ab5-4428-aa3b-18735d8d5e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:14.745006 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1051/2079] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:24:14 2022] DELETE /identity/v3/projects/045b3dd97b0c42f1b639ca09ba6adbc4 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:14.750509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19e12-0e89-4004-819b-6b0766912ec9 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:14.751012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19e12-0e89-4004-819b-6b0766912ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:14.751012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd19e12-0e89-4004-819b-6b0766912ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:14.780726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-ffd19e12-0e89-4004-819b-6b0766912ec9 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:14.782071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ffd19e12-0e89-4004-819b-6b0766912ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:14.791847 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:14.848633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1029/2080] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:14.853390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f88e6-2fd2-4989-a04a-335bd0aa1c38 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:14.853761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f88e6-2fd2-4989-a04a-335bd0aa1c38 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:14.854046 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f88e6-2fd2-4989-a04a-335bd0aa1c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:14.873905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-773f88e6-2fd2-4989-a04a-335bd0aa1c38 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:14.875120 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-773f88e6-2fd2-4989-a04a-335bd0aa1c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:14.883314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1052/2081] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:24:14.887803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd19e12-0e89-4004-819b-6b0766912ec9 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:14.898312 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd19e12-0e89-4004-819b-6b0766912ec9 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:14.904134 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb5b23-e36d-420f-87ee-145f3e7ada96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:14.904979 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb5b23-e36d-420f-87ee-145f3e7ada96 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:14.905367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb5b23-e36d-420f-87ee-145f3e7ada96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:14.905748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb5b23-e36d-420f-87ee-145f3e7ada96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:14.906609 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb5b23-e36d-420f-87ee-145f3e7ada96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:24:14.906923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb5b23-e36d-420f-87ee-145f3e7ada96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:14.908542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb5b23-e36d-420f-87ee-145f3e7ada96 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:14.923993 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1030/2082] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:24:14.931370 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f88e6-2fd2-4989-a04a-335bd0aa1c38 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:14.941879 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f88e6-2fd2-4989-a04a-335bd0aa1c38 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:14.946915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209f7a0-0b62-4ed6-8c18-c7958fe5a7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:14.947567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209f7a0-0b62-4ed6-8c18-c7958fe5a7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:14.947853 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209f7a0-0b62-4ed6-8c18-c7958fe5a7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:14.948131 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209f7a0-0b62-4ed6-8c18-c7958fe5a7a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:14.948820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209f7a0-0b62-4ed6-8c18-c7958fe5a7a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:14.950974 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209f7a0-0b62-4ed6-8c18-c7958fe5a7a2 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:15.047577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1053/2083] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:24:14 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:24:15.051775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb5b23-e36d-420f-87ee-145f3e7ada96 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:15.063185 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb5b23-e36d-420f-87ee-145f3e7ada96 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:15.125099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272724e7-3689-4a17-82dc-45b89c06757f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:15.125948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272724e7-3689-4a17-82dc-45b89c06757f admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:15.126390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272724e7-3689-4a17-82dc-45b89c06757f admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:15.126766 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272724e7-3689-4a17-82dc-45b89c06757f admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:15.127613 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272724e7-3689-4a17-82dc-45b89c06757f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:15.131530 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272724e7-3689-4a17-82dc-45b89c06757f admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:15.229552 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1031/2084] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:24:15 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:24:15.234616 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209f7a0-0b62-4ed6-8c18-c7958fe5a7a2 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:15.246220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209f7a0-0b62-4ed6-8c18-c7958fe5a7a2 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:15.251018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f72b4-8b08-4885-b95c-d7cd8c7811e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:15.251676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f72b4-8b08-4885-b95c-d7cd8c7811e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:15.251961 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f72b4-8b08-4885-b95c-d7cd8c7811e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:15.252238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f72b4-8b08-4885-b95c-d7cd8c7811e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:15.253053 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f72b4-8b08-4885-b95c-d7cd8c7811e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:15.254695 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f72b4-8b08-4885-b95c-d7cd8c7811e3 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:15.266478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1054/2085] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:24:15.272681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272724e7-3689-4a17-82dc-45b89c06757f admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:15.285160 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272724e7-3689-4a17-82dc-45b89c06757f admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:15.289997 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945e8fd-7b2b-4365-8de9-336cf7cf03db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:15.292085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945e8fd-7b2b-4365-8de9-336cf7cf03db admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:15.292085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945e8fd-7b2b-4365-8de9-336cf7cf03db admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:15.292085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945e8fd-7b2b-4365-8de9-336cf7cf03db admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:15.292085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945e8fd-7b2b-4365-8de9-336cf7cf03db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:15.294176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945e8fd-7b2b-4365-8de9-336cf7cf03db admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:15.306940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1032/2086] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:24:15.311668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f72b4-8b08-4885-b95c-d7cd8c7811e3 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:15.322279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f72b4-8b08-4885-b95c-d7cd8c7811e3 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:15.327224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9f579-c6fb-47a9-944f-4bedb437c5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:15.328114 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9f579-c6fb-47a9-944f-4bedb437c5cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:15.328392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9f579-c6fb-47a9-944f-4bedb437c5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:15.328665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e9f579-c6fb-47a9-944f-4bedb437c5cc admin admin] PATH_INFO: `/v3/projects/492be26275404bc3a439ae049bf4b9f4/users/abbf3822f4a6415b964112a383ad5509/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:15.344986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e9f579-c6fb-47a9-944f-4bedb437c5cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=492be26275404bc3a439ae049bf4b9f4, user_id=abbf3822f4a6415b964112a383ad5509, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:15.347949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e9f579-c6fb-47a9-944f-4bedb437c5cc admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:15.423819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1055/2087] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:24:15 2022] PUT /identity/v3/projects/492be26275404bc3a439ae049bf4b9f4/users/abbf3822f4a6415b964112a383ad5509/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:15.429220 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a5c86-0859-4aef-9747-4791057d926a None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:15.429513 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a5c86-0859-4aef-9747-4791057d926a None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:15.429972 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a5c86-0859-4aef-9747-4791057d926a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:15.450944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-199a5c86-0859-4aef-9747-4791057d926a None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:15.452187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-199a5c86-0859-4aef-9747-4791057d926a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:15.516646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1033/2088] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:15.527126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9f579-c6fb-47a9-944f-4bedb437c5cc admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:15.529401 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c6e9f579-c6fb-47a9-944f-4bedb437c5cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:15.552866 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:15.588090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e9f579-c6fb-47a9-944f-4bedb437c5cc admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:15.593137 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19602527-8902-45cb-b2d7-5fdf5926e781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:15.593790 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19602527-8902-45cb-b2d7-5fdf5926e781 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:15.594079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19602527-8902-45cb-b2d7-5fdf5926e781 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:15.594382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19602527-8902-45cb-b2d7-5fdf5926e781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:15.603894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19602527-8902-45cb-b2d7-5fdf5926e781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:15.605379 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19602527-8902-45cb-b2d7-5fdf5926e781 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:15.615790 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1056/2089] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:21.425223 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd7ffe-2550-4acb-9e8f-d2200d083b73 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:21.426121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd7ffe-2550-4acb-9e8f-d2200d083b73 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:21.426418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd7ffe-2550-4acb-9e8f-d2200d083b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:21.459819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-03dd7ffe-2550-4acb-9e8f-d2200d083b73 None None] MFA Rules not processed for user `abbf3822f4a6415b964112a383ad5509`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:21.461471 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-03dd7ffe-2550-4acb-9e8f-d2200d083b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:21.471084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:21.533947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1034/2090] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:21.538829 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19602527-8902-45cb-b2d7-5fdf5926e781 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:21.550115 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19602527-8902-45cb-b2d7-5fdf5926e781 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:21.554851 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2957f-4f46-42f0-9f9a-89f7440c297b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:21.555509 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2957f-4f46-42f0-9f9a-89f7440c297b admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:21.555792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2957f-4f46-42f0-9f9a-89f7440c297b admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:21.556069 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2957f-4f46-42f0-9f9a-89f7440c297b admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:21.556807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2957f-4f46-42f0-9f9a-89f7440c297b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:21.558772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2957f-4f46-42f0-9f9a-89f7440c297b admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:21.687506 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1057/2091] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:24:21 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:24:21.691773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd7ffe-2550-4acb-9e8f-d2200d083b73 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:21.703433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd7ffe-2550-4acb-9e8f-d2200d083b73 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:21.764630 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c903cf-4297-4be7-942b-72e6985fa4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:21.765341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c903cf-4297-4be7-942b-72e6985fa4c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:21.765629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c903cf-4297-4be7-942b-72e6985fa4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:21.765908 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c903cf-4297-4be7-942b-72e6985fa4c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:21.766619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c903cf-4297-4be7-942b-72e6985fa4c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:21.768583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c903cf-4297-4be7-942b-72e6985fa4c5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:21.840857 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1035/2092] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:24:21 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:24:21.845201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2957f-4f46-42f0-9f9a-89f7440c297b admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:21.855790 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2957f-4f46-42f0-9f9a-89f7440c297b admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:21.861355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2356af-5919-4848-9e62-bf30f89733c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:21.862079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2356af-5919-4848-9e62-bf30f89733c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:21.862368 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2356af-5919-4848-9e62-bf30f89733c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:21.862647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2356af-5919-4848-9e62-bf30f89733c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:21.863376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2356af-5919-4848-9e62-bf30f89733c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:21.864995 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2356af-5919-4848-9e62-bf30f89733c0 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:21.875526 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1058/2093] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:24:21.880625 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c903cf-4297-4be7-942b-72e6985fa4c5 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:21.891141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c903cf-4297-4be7-942b-72e6985fa4c5 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:21.895801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540cb94-c02c-4bc6-8318-8a88f2ed6042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:21.896636 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540cb94-c02c-4bc6-8318-8a88f2ed6042 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:21.896923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540cb94-c02c-4bc6-8318-8a88f2ed6042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:21.897234 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540cb94-c02c-4bc6-8318-8a88f2ed6042 admin admin] PATH_INFO: `/v3/projects/eb0111e05bcb4e029aa001ef1aec20b4/users/eddfaa0c35784e91b39251b4e7c1751c/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:21.913272 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540cb94-c02c-4bc6-8318-8a88f2ed6042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb0111e05bcb4e029aa001ef1aec20b4, user_id=eddfaa0c35784e91b39251b4e7c1751c, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:21.915905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540cb94-c02c-4bc6-8318-8a88f2ed6042 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:21.984318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1036/2094] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:24:21 2022] PUT /identity/v3/projects/eb0111e05bcb4e029aa001ef1aec20b4/users/eddfaa0c35784e91b39251b4e7c1751c/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:21.988433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2356af-5919-4848-9e62-bf30f89733c0 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:21.999503 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2356af-5919-4848-9e62-bf30f89733c0 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:22.067834 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb6e82-a4af-463d-bb5d-7fd5adbf3706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:22.068731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb6e82-a4af-463d-bb5d-7fd5adbf3706 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:22.069049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb6e82-a4af-463d-bb5d-7fd5adbf3706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:22.069335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb6e82-a4af-463d-bb5d-7fd5adbf3706 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:22.070107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb6e82-a4af-463d-bb5d-7fd5adbf3706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:22.073116 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fb6e82-a4af-463d-bb5d-7fd5adbf3706 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:22.095161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1059/2095] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:24:22.096388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540cb94-c02c-4bc6-8318-8a88f2ed6042 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:22.110192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540cb94-c02c-4bc6-8318-8a88f2ed6042 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:22.115964 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76f994-7cc2-4eca-9f8a-eb740906fcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:22.116877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76f994-7cc2-4eca-9f8a-eb740906fcb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:22.117199 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76f994-7cc2-4eca-9f8a-eb740906fcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:22.117481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76f994-7cc2-4eca-9f8a-eb740906fcb0 admin admin] PATH_INFO: `/v3/projects/eb0111e05bcb4e029aa001ef1aec20b4/users/eddfaa0c35784e91b39251b4e7c1751c/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:22.119555 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76f994-7cc2-4eca-9f8a-eb740906fcb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb0111e05bcb4e029aa001ef1aec20b4, user_id=eddfaa0c35784e91b39251b4e7c1751c, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:22.123220 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76f994-7cc2-4eca-9f8a-eb740906fcb0 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:22.136526 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1037/2096] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:24:22 2022] PUT /identity/v3/projects/eb0111e05bcb4e029aa001ef1aec20b4/users/eddfaa0c35784e91b39251b4e7c1751c/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:27.634979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb6e82-a4af-463d-bb5d-7fd5adbf3706 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:27.638926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-70fb6e82-a4af-463d-bb5d-7fd5adbf3706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:27.654028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fb6e82-a4af-463d-bb5d-7fd5adbf3706 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:27.747607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacec08-11dc-4bd4-a923-226bce0410c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:27.748829 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacec08-11dc-4bd4-a923-226bce0410c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:27.749330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacec08-11dc-4bd4-a923-226bce0410c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:27.749718 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcacec08-11dc-4bd4-a923-226bce0410c8 admin admin] PATH_INFO: `/v3/users/b21628dc64aa447d8234ef7848adbb18` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:27.752024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacec08-11dc-4bd4-a923-226bce0410c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b21628dc64aa447d8234ef7848adbb18)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:27.754383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcacec08-11dc-4bd4-a923-226bce0410c8 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:27.894704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:28.002185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-fcacec08-11dc-4bd4-a923-226bce0410c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b21628dc64aa447d8234ef7848adbb18'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:28.030005 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-fcacec08-11dc-4bd4-a923-226bce0410c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b21628dc64aa447d8234ef7848adbb18'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:28.131227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-fcacec08-11dc-4bd4-a923-226bce0410c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b21628dc64aa447d8234ef7848adbb18'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:28.144605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-fcacec08-11dc-4bd4-a923-226bce0410c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b21628dc64aa447d8234ef7848adbb18'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:28.145834 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-fcacec08-11dc-4bd4-a923-226bce0410c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b21628dc64aa447d8234ef7848adbb18'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:28.153844 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1060/2097] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:24:27 2022] DELETE /identity/v3/users/b21628dc64aa447d8234ef7848adbb18 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:28.159007 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839e7b9-7bd1-4ab3-9941-98461fd9a98f None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:28.163415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839e7b9-7bd1-4ab3-9941-98461fd9a98f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:28.163415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839e7b9-7bd1-4ab3-9941-98461fd9a98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:28.181257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-6839e7b9-7bd1-4ab3-9941-98461fd9a98f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:28.183200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-6839e7b9-7bd1-4ab3-9941-98461fd9a98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:28.247833 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacec08-11dc-4bd4-a923-226bce0410c8 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:28.249659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-fcacec08-11dc-4bd4-a923-226bce0410c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:28.337521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1038/2098] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:28.362281 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839e7b9-7bd1-4ab3-9941-98461fd9a98f None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:28.362281 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-6839e7b9-7bd1-4ab3-9941-98461fd9a98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:28.366984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:28.397219 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcacec08-11dc-4bd4-a923-226bce0410c8 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:28.410713 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4185bee-ad42-4453-8c42-d4c96e7642cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:28.411428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4185bee-ad42-4453-8c42-d4c96e7642cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:28.411762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4185bee-ad42-4453-8c42-d4c96e7642cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:28.412036 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4185bee-ad42-4453-8c42-d4c96e7642cd admin admin] PATH_INFO: `/v3/users/2de05736c1234a34a7792621309a0658` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:28.415267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4185bee-ad42-4453-8c42-d4c96e7642cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2de05736c1234a34a7792621309a0658)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:28.426068 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4185bee-ad42-4453-8c42-d4c96e7642cd admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:28.456147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839e7b9-7bd1-4ab3-9941-98461fd9a98f None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:28.460324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d844d8-573b-499b-83eb-fe1b70b1f457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:28.461112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d844d8-573b-499b-83eb-fe1b70b1f457 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:28.461399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d844d8-573b-499b-83eb-fe1b70b1f457 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:28.461674 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d844d8-573b-499b-83eb-fe1b70b1f457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:28.473619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d844d8-573b-499b-83eb-fe1b70b1f457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:28.475126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d844d8-573b-499b-83eb-fe1b70b1f457 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:28.487506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1039/2099] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:24:28.761129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c4185bee-ad42-4453-8c42-d4c96e7642cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2de05736c1234a34a7792621309a0658'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:28.780163 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c4185bee-ad42-4453-8c42-d4c96e7642cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2de05736c1234a34a7792621309a0658'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:28.875330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c4185bee-ad42-4453-8c42-d4c96e7642cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2de05736c1234a34a7792621309a0658'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:28.886460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c4185bee-ad42-4453-8c42-d4c96e7642cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2de05736c1234a34a7792621309a0658'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:28.889610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c4185bee-ad42-4453-8c42-d4c96e7642cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2de05736c1234a34a7792621309a0658'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:28.910898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1061/2100] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:24:28 2022] DELETE /identity/v3/users/2de05736c1234a34a7792621309a0658 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:24:28.916404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769f57c-2b58-4a29-910c-a671667589dd None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:28.916661 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769f57c-2b58-4a29-910c-a671667589dd None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:28.916946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769f57c-2b58-4a29-910c-a671667589dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:28.952503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-5769f57c-2b58-4a29-910c-a671667589dd None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:28.958041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-5769f57c-2b58-4a29-910c-a671667589dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:29.074728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1040/2101] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:29.089103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4185bee-ad42-4453-8c42-d4c96e7642cd admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:29.095125 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c4185bee-ad42-4453-8c42-d4c96e7642cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:29.195765 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4185bee-ad42-4453-8c42-d4c96e7642cd admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:29.200665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0e9bd-5867-441a-8ab0-a745ead04a46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:29.215112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0e9bd-5867-441a-8ab0-a745ead04a46 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:29.215112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0e9bd-5867-441a-8ab0-a745ead04a46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:29.215112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0e9bd-5867-441a-8ab0-a745ead04a46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:29.230532 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0e9bd-5867-441a-8ab0-a745ead04a46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:29.232448 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0e9bd-5867-441a-8ab0-a745ead04a46 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:29.245122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1062/2102] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:29.287792 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769f57c-2b58-4a29-910c-a671667589dd None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:29.289444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-5769f57c-2b58-4a29-910c-a671667589dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:29.310149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769f57c-2b58-4a29-910c-a671667589dd None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:29.344090 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a661de5-5562-4e31-93ea-fef905859c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:29.345372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a661de5-5562-4e31-93ea-fef905859c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:29.345682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a661de5-5562-4e31-93ea-fef905859c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:29.345962 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a661de5-5562-4e31-93ea-fef905859c57 admin admin] PATH_INFO: `/v3/projects/b2f44e31260240558eab4143262c3978` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:29.349200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a661de5-5562-4e31-93ea-fef905859c57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2f44e31260240558eab4143262c3978)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:29.351390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a661de5-5562-4e31-93ea-fef905859c57 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:29.423671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:29.592070 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6a661de5-5562-4e31-93ea-fef905859c57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2f44e31260240558eab4143262c3978', 'request_id': 'req-6a661de5-5562-4e31-93ea-fef905859c57'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:29.603271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6a661de5-5562-4e31-93ea-fef905859c57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2f44e31260240558eab4143262c3978', 'request_id': 'req-6a661de5-5562-4e31-93ea-fef905859c57'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:29.726695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6a661de5-5562-4e31-93ea-fef905859c57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2f44e31260240558eab4143262c3978', 'request_id': 'req-6a661de5-5562-4e31-93ea-fef905859c57'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:29.742443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6a661de5-5562-4e31-93ea-fef905859c57 admin admin] The token cache is being invalidate because project b2f44e31260240558eab4143262c3978 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:24:29.742787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6a661de5-5562-4e31-93ea-fef905859c57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:29.743481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6a661de5-5562-4e31-93ea-fef905859c57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:29.745528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1041/2103] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:24:29 2022] DELETE /identity/v3/projects/b2f44e31260240558eab4143262c3978 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:29.751477 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c8e83-1cee-417b-ab22-e23c6c819585 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:29.754330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c8e83-1cee-417b-ab22-e23c6c819585 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:29.754330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c8e83-1cee-417b-ab22-e23c6c819585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:29.836377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-469c8e83-1cee-417b-ab22-e23c6c819585 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:29.837660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-469c8e83-1cee-417b-ab22-e23c6c819585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:29.932662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1063/2104] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:29.941063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7671900-ac4b-4860-9f97-48003c854728 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:29.943845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7671900-ac4b-4860-9f97-48003c854728 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:29.947409 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7671900-ac4b-4860-9f97-48003c854728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:29.971362 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-d7671900-ac4b-4860-9f97-48003c854728 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:29.972717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d7671900-ac4b-4860-9f97-48003c854728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:29.981534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1042/2105] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:29.986204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c8e83-1cee-417b-ab22-e23c6c819585 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:29.998127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c8e83-1cee-417b-ab22-e23c6c819585 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:30.000915 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7671900-ac4b-4860-9f97-48003c854728 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:30.003720 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d7671900-ac4b-4860-9f97-48003c854728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:30.004401 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ca9e2-caf2-4ac4-aab1-5afaa34ddc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:30.005656 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ca9e2-caf2-4ac4-aab1-5afaa34ddc9b admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:30.006437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ca9e2-caf2-4ac4-aab1-5afaa34ddc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:30.006910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ca9e2-caf2-4ac4-aab1-5afaa34ddc9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:30.007936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ca9e2-caf2-4ac4-aab1-5afaa34ddc9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:24:30.008458 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ca9e2-caf2-4ac4-aab1-5afaa34ddc9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:30.010157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ca9e2-caf2-4ac4-aab1-5afaa34ddc9b admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:30.031897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7671900-ac4b-4860-9f97-48003c854728 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:30.036881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146bd7e-c57e-4b52-a881-009278c8e233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:30.037925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146bd7e-c57e-4b52-a881-009278c8e233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:30.038283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146bd7e-c57e-4b52-a881-009278c8e233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:30.038563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146bd7e-c57e-4b52-a881-009278c8e233 admin admin] PATH_INFO: `/v3/projects/2cf7398c2e3846eba51beeaacddfb685` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:30.040982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146bd7e-c57e-4b52-a881-009278c8e233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cf7398c2e3846eba51beeaacddfb685)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:30.043196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146bd7e-c57e-4b52-a881-009278c8e233 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:30.065701 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1064/2106] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:24:30.066120 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ca9e2-caf2-4ac4-aab1-5afaa34ddc9b admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:30.085818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ca9e2-caf2-4ac4-aab1-5afaa34ddc9b admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:30.091785 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:30.101290 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b50e9c-023a-4a4f-889d-b565713aee80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:30.101961 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b50e9c-023a-4a4f-889d-b565713aee80 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:30.102253 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b50e9c-023a-4a4f-889d-b565713aee80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:30.102534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b50e9c-023a-4a4f-889d-b565713aee80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:30.103297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b50e9c-023a-4a4f-889d-b565713aee80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:30.105186 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b50e9c-023a-4a4f-889d-b565713aee80 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:30.182931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-a146bd7e-c57e-4b52-a881-009278c8e233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cf7398c2e3846eba51beeaacddfb685', 'request_id': 'req-a146bd7e-c57e-4b52-a881-009278c8e233'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:30.189402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-a146bd7e-c57e-4b52-a881-009278c8e233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cf7398c2e3846eba51beeaacddfb685', 'request_id': 'req-a146bd7e-c57e-4b52-a881-009278c8e233'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:30.204238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1065/2107] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:24:30 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:24:30.205885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f22dae-6b2d-4520-b05d-270aca76a1e6 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:30.206492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f22dae-6b2d-4520-b05d-270aca76a1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:30.206800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f22dae-6b2d-4520-b05d-270aca76a1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:30.235862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-b9f22dae-6b2d-4520-b05d-270aca76a1e6 None None] MFA Rules not processed for user `eddfaa0c35784e91b39251b4e7c1751c`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:30.237189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b9f22dae-6b2d-4520-b05d-270aca76a1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:30.257722 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-a146bd7e-c57e-4b52-a881-009278c8e233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cf7398c2e3846eba51beeaacddfb685', 'request_id': 'req-a146bd7e-c57e-4b52-a881-009278c8e233'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:30.287801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-a146bd7e-c57e-4b52-a881-009278c8e233 admin admin] The token cache is being invalidate because project 2cf7398c2e3846eba51beeaacddfb685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:24:30.288296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-a146bd7e-c57e-4b52-a881-009278c8e233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:30.290146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-a146bd7e-c57e-4b52-a881-009278c8e233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:30.291935 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1043/2108] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:24:29 2022] DELETE /identity/v3/projects/2cf7398c2e3846eba51beeaacddfb685 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:30.293080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146bd7e-c57e-4b52-a881-009278c8e233 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:30.295361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a146bd7e-c57e-4b52-a881-009278c8e233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:30.319416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1066/2109] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:30.321537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec182a3-0ab2-409b-ba58-84ed510db5ce None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:30.321651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec182a3-0ab2-409b-ba58-84ed510db5ce None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:30.322492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec182a3-0ab2-409b-ba58-84ed510db5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:30.343075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-4ec182a3-0ab2-409b-ba58-84ed510db5ce None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:30.344387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-4ec182a3-0ab2-409b-ba58-84ed510db5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:30.382053 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146bd7e-c57e-4b52-a881-009278c8e233 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:30.414114 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1067/2110] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:30.414114 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec182a3-0ab2-409b-ba58-84ed510db5ce None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:30.414114 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-4ec182a3-0ab2-409b-ba58-84ed510db5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:30.456502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518cb5d-6be5-40f5-85da-66b8454ab65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:30.457233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518cb5d-6be5-40f5-85da-66b8454ab65c admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:30.457572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518cb5d-6be5-40f5-85da-66b8454ab65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:30.457774 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518cb5d-6be5-40f5-85da-66b8454ab65c admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:30.458476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518cb5d-6be5-40f5-85da-66b8454ab65c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:30.461127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518cb5d-6be5-40f5-85da-66b8454ab65c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:30.550145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec182a3-0ab2-409b-ba58-84ed510db5ce None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:30.556724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b42764-36ca-4c3d-b12b-5800a0c506b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:30.557016 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1044/2111] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:24:30 2022] POST /identity/v3/users => generated 286 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:24:30.557243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b42764-36ca-4c3d-b12b-5800a0c506b9 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:30.557536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b42764-36ca-4c3d-b12b-5800a0c506b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:30.557816 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b42764-36ca-4c3d-b12b-5800a0c506b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:30.558662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9072e93-8aa9-4e97-83b3-012e7b56f1ed None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:30.559022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9072e93-8aa9-4e97-83b3-012e7b56f1ed None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:30.559796 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9072e93-8aa9-4e97-83b3-012e7b56f1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:30.560653 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-94b42764-36ca-4c3d-b12b-5800a0c506b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:30.581149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a9072e93-8aa9-4e97-83b3-012e7b56f1ed None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:30.582618 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a9072e93-8aa9-4e97-83b3-012e7b56f1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:30.592919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1045/2112] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:30.595210 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9072e93-8aa9-4e97-83b3-012e7b56f1ed None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:30.608082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9072e93-8aa9-4e97-83b3-012e7b56f1ed None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:30.613745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c538a-065d-42c9-8fca-cf45818ec046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:30.613745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c538a-065d-42c9-8fca-cf45818ec046 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:30.614019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c538a-065d-42c9-8fca-cf45818ec046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:30.614244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c538a-065d-42c9-8fca-cf45818ec046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:30.615174 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c538a-065d-42c9-8fca-cf45818ec046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:30.617439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c538a-065d-42c9-8fca-cf45818ec046 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:30.633787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1046/2113] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:24:30.633787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c538a-065d-42c9-8fca-cf45818ec046 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:30.640744 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b42764-36ca-4c3d-b12b-5800a0c506b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:30.642597 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b42764-36ca-4c3d-b12b-5800a0c506b9 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:30.642919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c538a-065d-42c9-8fca-cf45818ec046 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:30.648412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821fceb7-9c42-45c9-a526-5fe6052ef359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:30.648934 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fceb7-9c42-45c9-a526-5fe6052ef359 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:30.652423 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fceb7-9c42-45c9-a526-5fe6052ef359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:30.652423 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fceb7-9c42-45c9-a526-5fe6052ef359 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:30.652423 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821fceb7-9c42-45c9-a526-5fe6052ef359 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:24:30.652423 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821fceb7-9c42-45c9-a526-5fe6052ef359 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:30.652777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:30.652864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821fceb7-9c42-45c9-a526-5fe6052ef359 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:30.656197 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1068/2114] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:30.658241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b42764-36ca-4c3d-b12b-5800a0c506b9 service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:30.670103 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1047/2115] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:24:30.673315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b42764-36ca-4c3d-b12b-5800a0c506b9 service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:30.674532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821fceb7-9c42-45c9-a526-5fe6052ef359 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:30.680465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e523934-d3cb-49a1-8b48-d54452a8b0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:30.681184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e523934-d3cb-49a1-8b48-d54452a8b0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:30.681472 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e523934-d3cb-49a1-8b48-d54452a8b0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:30.681755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e523934-d3cb-49a1-8b48-d54452a8b0c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:30.682803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e523934-d3cb-49a1-8b48-d54452a8b0c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:30.686381 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e523934-d3cb-49a1-8b48-d54452a8b0c7 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:30.689409 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821fceb7-9c42-45c9-a526-5fe6052ef359 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:30.701296 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1069/2116] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:24:30.706233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e523934-d3cb-49a1-8b48-d54452a8b0c7 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:30.713066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842712b6-d482-4cf4-9adb-cabfb06c594d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:30.715221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842712b6-d482-4cf4-9adb-cabfb06c594d admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:30.716658 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842712b6-d482-4cf4-9adb-cabfb06c594d admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:30.717335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842712b6-d482-4cf4-9adb-cabfb06c594d admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:30.718542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842712b6-d482-4cf4-9adb-cabfb06c594d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:30.721276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842712b6-d482-4cf4-9adb-cabfb06c594d admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:30.724141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e523934-d3cb-49a1-8b48-d54452a8b0c7 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:30.730436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aba4fd-8d1d-42d2-8ff2-f13f0ad83a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:30.731513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aba4fd-8d1d-42d2-8ff2-f13f0ad83a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:30.731800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aba4fd-8d1d-42d2-8ff2-f13f0ad83a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:30.732082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aba4fd-8d1d-42d2-8ff2-f13f0ad83a33 admin admin] PATH_INFO: `/v3/projects/75f8396469a743d7a9b34f228b53e3cd/users/0212b3d4ad654667ab2f896d63919242/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:30.749670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aba4fd-8d1d-42d2-8ff2-f13f0ad83a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f8396469a743d7a9b34f228b53e3cd, user_id=0212b3d4ad654667ab2f896d63919242, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:30.752708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aba4fd-8d1d-42d2-8ff2-f13f0ad83a33 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:30.835440 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1048/2117] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:24:30 2022] POST /identity/v3/projects => generated 381 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:24:30.839706 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842712b6-d482-4cf4-9adb-cabfb06c594d admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:30.854516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1070/2118] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:24:30 2022] PUT /identity/v3/projects/75f8396469a743d7a9b34f228b53e3cd/users/0212b3d4ad654667ab2f896d63919242/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:30.868867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4dc4f8-875d-43c0-a057-48f2a363f1b4 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:30.869323 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4dc4f8-875d-43c0-a057-48f2a363f1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:30.869736 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4dc4f8-875d-43c0-a057-48f2a363f1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:30.915775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-8d4dc4f8-875d-43c0-a057-48f2a363f1b4 None None] MFA Rules not processed for user `0212b3d4ad654667ab2f896d63919242`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:30.917095 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-8d4dc4f8-875d-43c0-a057-48f2a363f1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:30.927363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842712b6-d482-4cf4-9adb-cabfb06c594d admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:30.935814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:30.964644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:31.000207 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19cfb8-7a3a-49f8-9105-606435d836c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:31.000914 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19cfb8-7a3a-49f8-9105-606435d836c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:31.001230 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19cfb8-7a3a-49f8-9105-606435d836c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:31.001524 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19cfb8-7a3a-49f8-9105-606435d836c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:31.002285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd19cfb8-7a3a-49f8-9105-606435d836c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:31.004348 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd19cfb8-7a3a-49f8-9105-606435d836c4 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:31.023628 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1071/2119] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:24:31.042762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4dc4f8-875d-43c0-a057-48f2a363f1b4 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:31.045370 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-8d4dc4f8-875d-43c0-a057-48f2a363f1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:31.111065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1049/2120] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:24:30 2022] POST /identity/v3/users => generated 291 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:24:31.115687 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19cfb8-7a3a-49f8-9105-606435d836c4 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:31.120282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4dc4f8-875d-43c0-a057-48f2a363f1b4 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:31.125887 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb64cee-f6d7-4770-8a4e-c2423b57e3b0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fbf33189aa94f4f973d558d615a23e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:31.126564 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb64cee-f6d7-4770-8a4e-c2423b57e3b0 service glance] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:31.126855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb64cee-f6d7-4770-8a4e-c2423b57e3b0 service glance] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:31.127137 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb64cee-f6d7-4770-8a4e-c2423b57e3b0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:31.127479 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19cfb8-7a3a-49f8-9105-606435d836c4 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:31.133886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4beb6-058b-40da-9e6c-e9a84eb10260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:31.135019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4beb6-058b-40da-9e6c-e9a84eb10260 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:31.135434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4beb6-058b-40da-9e6c-e9a84eb10260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:31.135873 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4beb6-058b-40da-9e6c-e9a84eb10260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:31.136598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4beb6-058b-40da-9e6c-e9a84eb10260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:31.137696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb64cee-f6d7-4770-8a4e-c2423b57e3b0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:31.139082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb64cee-f6d7-4770-8a4e-c2423b57e3b0 service glance] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:31.139226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4beb6-058b-40da-9e6c-e9a84eb10260 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:31.150648 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1072/2121] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:24:31.154697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1050/2122] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:24:31.161861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb64cee-f6d7-4770-8a4e-c2423b57e3b0 service glance] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:31.174796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb64cee-f6d7-4770-8a4e-c2423b57e3b0 service glance] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:31.185253 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512405f-89d5-4235-a8a5-002924c2673e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:31.185702 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512405f-89d5-4235-a8a5-002924c2673e admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:31.186757 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512405f-89d5-4235-a8a5-002924c2673e admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:31.187049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512405f-89d5-4235-a8a5-002924c2673e admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:31.189533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512405f-89d5-4235-a8a5-002924c2673e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:31.191392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512405f-89d5-4235-a8a5-002924c2673e admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:31.204567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1073/2123] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:24:31.208903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4beb6-058b-40da-9e6c-e9a84eb10260 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:31.221757 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4beb6-058b-40da-9e6c-e9a84eb10260 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:31.227644 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b389b-7fe7-4bc3-bc6a-ffa2e1bc17a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:31.228505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b389b-7fe7-4bc3-bc6a-ffa2e1bc17a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:31.228784 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b389b-7fe7-4bc3-bc6a-ffa2e1bc17a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:31.229100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b389b-7fe7-4bc3-bc6a-ffa2e1bc17a5 admin admin] PATH_INFO: `/v3/projects/c04aac52296b46649c8e344aa9e0fa92/users/58bdd7080ad04320b4d96d944b89cc1a/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:31.257437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b389b-7fe7-4bc3-bc6a-ffa2e1bc17a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c04aac52296b46649c8e344aa9e0fa92, user_id=58bdd7080ad04320b4d96d944b89cc1a, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:31.260154 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b389b-7fe7-4bc3-bc6a-ffa2e1bc17a5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:31.359117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1051/2124] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:24:31 2022] PUT /identity/v3/projects/c04aac52296b46649c8e344aa9e0fa92/users/58bdd7080ad04320b4d96d944b89cc1a/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:31.369876 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26dfb6-8b33-4cb3-97d7-f376051a8e97 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:31.370172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26dfb6-8b33-4cb3-97d7-f376051a8e97 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:31.370504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26dfb6-8b33-4cb3-97d7-f376051a8e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:31.404884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-8e26dfb6-8b33-4cb3-97d7-f376051a8e97 None None] MFA Rules not processed for user `58bdd7080ad04320b4d96d944b89cc1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:31.406179 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-8e26dfb6-8b33-4cb3-97d7-f376051a8e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:31.506783 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1074/2125] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:24:31.511743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b389b-7fe7-4bc3-bc6a-ffa2e1bc17a5 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:31.586115 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:31.607698 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b389b-7fe7-4bc3-bc6a-ffa2e1bc17a5 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:31.614519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698f968-651c-4ceb-b7ed-586f5023c512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:31.615251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698f968-651c-4ceb-b7ed-586f5023c512 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:31.615544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698f968-651c-4ceb-b7ed-586f5023c512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:31.615852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698f968-651c-4ceb-b7ed-586f5023c512 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:31.616572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698f968-651c-4ceb-b7ed-586f5023c512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:31.618599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698f968-651c-4ceb-b7ed-586f5023c512 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:31.745398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1052/2126] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:24:31 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:24:31.750018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26dfb6-8b33-4cb3-97d7-f376051a8e97 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:31.756831 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:31.761112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26dfb6-8b33-4cb3-97d7-f376051a8e97 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:31.817070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadea802-061b-497b-b07f-6e25141d2ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:31.817763 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadea802-061b-497b-b07f-6e25141d2ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:31.818056 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadea802-061b-497b-b07f-6e25141d2ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:31.818340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadea802-061b-497b-b07f-6e25141d2ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:31.819052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadea802-061b-497b-b07f-6e25141d2ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:31.821266 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadea802-061b-497b-b07f-6e25141d2ff9 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:31.953014 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1075/2127] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:24:31 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:24:31.955737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698f968-651c-4ceb-b7ed-586f5023c512 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:31.972174 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698f968-651c-4ceb-b7ed-586f5023c512 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:31.977433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0351cc-f221-4df3-9f90-f7b3b29d6484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:31.978040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0351cc-f221-4df3-9f90-f7b3b29d6484 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:31.978323 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0351cc-f221-4df3-9f90-f7b3b29d6484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:31.978736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0351cc-f221-4df3-9f90-f7b3b29d6484 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:31.979494 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0351cc-f221-4df3-9f90-f7b3b29d6484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:31.981492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0351cc-f221-4df3-9f90-f7b3b29d6484 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:32.002798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1053/2128] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:24:32.006766 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadea802-061b-497b-b07f-6e25141d2ff9 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:32.018070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadea802-061b-497b-b07f-6e25141d2ff9 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:32.026498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08843391-c932-4ef6-96d5-d369d8e0a8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:32.027453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08843391-c932-4ef6-96d5-d369d8e0a8db admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:32.033181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08843391-c932-4ef6-96d5-d369d8e0a8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:32.033181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08843391-c932-4ef6-96d5-d369d8e0a8db admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:32.033181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08843391-c932-4ef6-96d5-d369d8e0a8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:32.033181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08843391-c932-4ef6-96d5-d369d8e0a8db admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:32.052153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0351cc-f221-4df3-9f90-f7b3b29d6484 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:32.053191 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1076/2129] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:24:32.065196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0351cc-f221-4df3-9f90-f7b3b29d6484 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:32.070592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858d983-52a4-4ee0-b282-32c9bc15b8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:32.071779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858d983-52a4-4ee0-b282-32c9bc15b8ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:32.072068 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858d983-52a4-4ee0-b282-32c9bc15b8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:32.072351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858d983-52a4-4ee0-b282-32c9bc15b8ef admin admin] PATH_INFO: `/v3/projects/c8dff9e711094ebbb967b197f525cedd/users/1fcef1152767492a9a2de847eab60d7b/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:32.090983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858d983-52a4-4ee0-b282-32c9bc15b8ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8dff9e711094ebbb967b197f525cedd, user_id=1fcef1152767492a9a2de847eab60d7b, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:32.099449 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858d983-52a4-4ee0-b282-32c9bc15b8ef admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:32.229395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1054/2130] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:24:32 2022] PUT /identity/v3/projects/c8dff9e711094ebbb967b197f525cedd/users/1fcef1152767492a9a2de847eab60d7b/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:24:32.240375 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849c4bc-9edd-463e-980e-3cd049f558a2 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:32.240664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849c4bc-9edd-463e-980e-3cd049f558a2 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:32.240950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849c4bc-9edd-463e-980e-3cd049f558a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:32.268899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-3849c4bc-9edd-463e-980e-3cd049f558a2 None None] MFA Rules not processed for user `1fcef1152767492a9a2de847eab60d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:32.270222 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-3849c4bc-9edd-463e-980e-3cd049f558a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:32.401292 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1077/2131] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:32.410681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858d983-52a4-4ee0-b282-32c9bc15b8ef admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:32.534671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858d983-52a4-4ee0-b282-32c9bc15b8ef admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:32.540017 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457ee34-4615-4293-9acb-11b3a15b955e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:32.541204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457ee34-4615-4293-9acb-11b3a15b955e admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:32.545511 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457ee34-4615-4293-9acb-11b3a15b955e admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:32.546046 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457ee34-4615-4293-9acb-11b3a15b955e admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:32.547096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457ee34-4615-4293-9acb-11b3a15b955e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:32.548920 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457ee34-4615-4293-9acb-11b3a15b955e admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:32.719414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1055/2132] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:24:32 2022] POST /identity/v3/projects => generated 385 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:24:32.723244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849c4bc-9edd-463e-980e-3cd049f558a2 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:32.735838 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849c4bc-9edd-463e-980e-3cd049f558a2 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:32.806710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd9e2b-d915-4614-9c82-dde5220c8836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:32.807382 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd9e2b-d915-4614-9c82-dde5220c8836 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:32.807668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd9e2b-d915-4614-9c82-dde5220c8836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:32.807943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd9e2b-d915-4614-9c82-dde5220c8836 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:32.808642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd9e2b-d915-4614-9c82-dde5220c8836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:32.811323 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd9e2b-d915-4614-9c82-dde5220c8836 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:32.897190 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1078/2133] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:24:32 2022] POST /identity/v3/users => generated 299 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:24:32.901374 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457ee34-4615-4293-9acb-11b3a15b955e admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:32.913776 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457ee34-4615-4293-9acb-11b3a15b955e admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:32.919581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bea3b7-4920-43cb-aca0-82b0954bb315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:32.920533 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bea3b7-4920-43cb-aca0-82b0954bb315 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:32.920852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bea3b7-4920-43cb-aca0-82b0954bb315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:32.921168 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bea3b7-4920-43cb-aca0-82b0954bb315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:32.921921 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bea3b7-4920-43cb-aca0-82b0954bb315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:32.923604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bea3b7-4920-43cb-aca0-82b0954bb315 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:32.937082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1056/2134] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:24:32.941349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd9e2b-d915-4614-9c82-dde5220c8836 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:32.946943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:32.953498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd9e2b-d915-4614-9c82-dde5220c8836 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:32.958434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82409ad1-4cbe-4fdc-98c2-d53084d67078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:32.959327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82409ad1-4cbe-4fdc-98c2-d53084d67078 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:32.959613 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82409ad1-4cbe-4fdc-98c2-d53084d67078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:32.959895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82409ad1-4cbe-4fdc-98c2-d53084d67078 admin admin] PATH_INFO: `/v3/projects/6c18cb936b034993b652db0f15dc729b/users/ab7adf70fce7421c96b679f94783f9d5/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:32.979444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82409ad1-4cbe-4fdc-98c2-d53084d67078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c18cb936b034993b652db0f15dc729b, user_id=ab7adf70fce7421c96b679f94783f9d5, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:32.982216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82409ad1-4cbe-4fdc-98c2-d53084d67078 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:33.099258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1079/2135] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:24:32 2022] PUT /identity/v3/projects/6c18cb936b034993b652db0f15dc729b/users/ab7adf70fce7421c96b679f94783f9d5/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:33.103904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bea3b7-4920-43cb-aca0-82b0954bb315 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:33.115000 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bea3b7-4920-43cb-aca0-82b0954bb315 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:33.201331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d686d-805d-44ab-8f20-f7e3657f038e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:33.202304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d686d-805d-44ab-8f20-f7e3657f038e admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:33.202304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d686d-805d-44ab-8f20-f7e3657f038e admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:33.202595 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d686d-805d-44ab-8f20-f7e3657f038e admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:33.203295 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d686d-805d-44ab-8f20-f7e3657f038e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:33.205016 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d686d-805d-44ab-8f20-f7e3657f038e admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:33.224070 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1057/2136] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:24:33.228160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82409ad1-4cbe-4fdc-98c2-d53084d67078 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:33.240319 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82409ad1-4cbe-4fdc-98c2-d53084d67078 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:33.245493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d7309-f8e7-4c09-a2c2-7c3b67558661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:33.246750 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d7309-f8e7-4c09-a2c2-7c3b67558661 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:33.247271 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d7309-f8e7-4c09-a2c2-7c3b67558661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:33.247693 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d7309-f8e7-4c09-a2c2-7c3b67558661 admin admin] PATH_INFO: `/v3/projects/6c18cb936b034993b652db0f15dc729b/users/ab7adf70fce7421c96b679f94783f9d5/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:33.250529 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d7309-f8e7-4c09-a2c2-7c3b67558661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c18cb936b034993b652db0f15dc729b, user_id=ab7adf70fce7421c96b679f94783f9d5, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:33.257662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d7309-f8e7-4c09-a2c2-7c3b67558661 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:33.273497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1080/2137] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:24:33 2022] PUT /identity/v3/projects/6c18cb936b034993b652db0f15dc729b/users/ab7adf70fce7421c96b679f94783f9d5/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:33.287950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c294c3-7ccd-4a1c-bb18-558e7f0faf65 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:33.288260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c294c3-7ccd-4a1c-bb18-558e7f0faf65 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:33.288511 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c294c3-7ccd-4a1c-bb18-558e7f0faf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:33.312146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:33.327721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-29c294c3-7ccd-4a1c-bb18-558e7f0faf65 None None] MFA Rules not processed for user `ab7adf70fce7421c96b679f94783f9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:33.329062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-29c294c3-7ccd-4a1c-bb18-558e7f0faf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:33.413354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1058/2138] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:33.433753 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d7309-f8e7-4c09-a2c2-7c3b67558661 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:33.435612 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-139d7309-f8e7-4c09-a2c2-7c3b67558661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:33.526387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d7309-f8e7-4c09-a2c2-7c3b67558661 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:33.530062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312fbab-1461-4dd5-b9fe-f2f78970ffa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:33.530732 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312fbab-1461-4dd5-b9fe-f2f78970ffa2 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:33.531015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312fbab-1461-4dd5-b9fe-f2f78970ffa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:33.531293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312fbab-1461-4dd5-b9fe-f2f78970ffa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:33.603000 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312fbab-1461-4dd5-b9fe-f2f78970ffa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:33.604371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312fbab-1461-4dd5-b9fe-f2f78970ffa2 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:33.615507 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1081/2139] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:34.052778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e5313-d85b-4ccd-9122-c5d64064f1e7 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:34.053106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e5313-d85b-4ccd-9122-c5d64064f1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:34.053391 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e5313-d85b-4ccd-9122-c5d64064f1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:34.072318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-320e5313-d85b-4ccd-9122-c5d64064f1e7 None None] MFA Rules not processed for user `abbf3822f4a6415b964112a383ad5509`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:34.073588 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-320e5313-d85b-4ccd-9122-c5d64064f1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:34.131124 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1059/2140] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:37.697945 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312fbab-1461-4dd5-b9fe-f2f78970ffa2 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:37.699536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9312fbab-1461-4dd5-b9fe-f2f78970ffa2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:37.711448 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312fbab-1461-4dd5-b9fe-f2f78970ffa2 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:37.715323 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49434a6-4957-4814-8357-7a08afd38ad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:37.716726 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49434a6-4957-4814-8357-7a08afd38ad8 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:37.717107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49434a6-4957-4814-8357-7a08afd38ad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:37.717401 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49434a6-4957-4814-8357-7a08afd38ad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:37.777053 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49434a6-4957-4814-8357-7a08afd38ad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:37.778646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49434a6-4957-4814-8357-7a08afd38ad8 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:37.782340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:37.788266 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1082/2141] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:39.284351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e5313-d85b-4ccd-9122-c5d64064f1e7 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:39.294473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e5313-d85b-4ccd-9122-c5d64064f1e7 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:39.298168 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97cdfb0-97a2-46fc-aba8-00c14eff674f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:39.299485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cdfb0-97a2-46fc-aba8-00c14eff674f service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:39.299485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cdfb0-97a2-46fc-aba8-00c14eff674f service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:39.299485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97cdfb0-97a2-46fc-aba8-00c14eff674f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:39.309108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97cdfb0-97a2-46fc-aba8-00c14eff674f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:39.311121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97cdfb0-97a2-46fc-aba8-00c14eff674f service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:39.321563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1060/2142] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:43.527041 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dbb7c-651b-4b2d-a94a-fe97cac1f9f6 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:43.527553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dbb7c-651b-4b2d-a94a-fe97cac1f9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:43.527669 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6dbb7c-651b-4b2d-a94a-fe97cac1f9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:43.557155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-9a6dbb7c-651b-4b2d-a94a-fe97cac1f9f6 None None] MFA Rules not processed for user `5090fa81be4d41bbb62da2c4f273d537`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:43.558232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9a6dbb7c-651b-4b2d-a94a-fe97cac1f9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:43.619853 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1083/2143] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:44.974339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97cdfb0-97a2-46fc-aba8-00c14eff674f service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:45.039505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97cdfb0-97a2-46fc-aba8-00c14eff674f service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:45.044325 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92db9a-6894-4817-b967-4c85024f40d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:45.045986 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92db9a-6894-4817-b967-4c85024f40d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:45.046293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92db9a-6894-4817-b967-4c85024f40d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:45.046563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92db9a-6894-4817-b967-4c85024f40d1 admin admin] PATH_INFO: `/v3/users/58bdd7080ad04320b4d96d944b89cc1a` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:45.047781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92db9a-6894-4817-b967-4c85024f40d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58bdd7080ad04320b4d96d944b89cc1a)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:45.049866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92db9a-6894-4817-b967-4c85024f40d1 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:45.228658 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:45.244331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-df92db9a-6894-4817-b967-4c85024f40d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58bdd7080ad04320b4d96d944b89cc1a'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:45.321095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-df92db9a-6894-4817-b967-4c85024f40d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58bdd7080ad04320b4d96d944b89cc1a'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:45.321344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-df92db9a-6894-4817-b967-4c85024f40d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58bdd7080ad04320b4d96d944b89cc1a'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:45.322267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-df92db9a-6894-4817-b967-4c85024f40d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58bdd7080ad04320b4d96d944b89cc1a'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:45.336019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-df92db9a-6894-4817-b967-4c85024f40d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58bdd7080ad04320b4d96d944b89cc1a'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:45.349860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1061/2144] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:24:44 2022] DELETE /identity/v3/users/58bdd7080ad04320b4d96d944b89cc1a => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:45.354407 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8beb591-1912-479b-baac-9ad7d0dbb596 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:45.354887 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8beb591-1912-479b-baac-9ad7d0dbb596 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:45.354978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8beb591-1912-479b-baac-9ad7d0dbb596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:45.375961 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-d8beb591-1912-479b-baac-9ad7d0dbb596 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:45.377218 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d8beb591-1912-479b-baac-9ad7d0dbb596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:45.443865 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1084/2145] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:24:45.456581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92db9a-6894-4817-b967-4c85024f40d1 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:45.458464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-df92db9a-6894-4817-b967-4c85024f40d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:45.519380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92db9a-6894-4817-b967-4c85024f40d1 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:45.523398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05cb434-7ae5-4cdf-89d9-91b834b0d01e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:45.524274 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05cb434-7ae5-4cdf-89d9-91b834b0d01e service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:45.524656 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05cb434-7ae5-4cdf-89d9-91b834b0d01e service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:45.525180 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05cb434-7ae5-4cdf-89d9-91b834b0d01e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:45.543868 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05cb434-7ae5-4cdf-89d9-91b834b0d01e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:45.545080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05cb434-7ae5-4cdf-89d9-91b834b0d01e service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:45.555694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1062/2146] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:46.019139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8beb591-1912-479b-baac-9ad7d0dbb596 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:46.021093 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d8beb591-1912-479b-baac-9ad7d0dbb596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:46.036455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8beb591-1912-479b-baac-9ad7d0dbb596 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:46.042157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:46.043090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:46.043342 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:46.043660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7 admin admin] PATH_INFO: `/v3/projects/c04aac52296b46649c8e344aa9e0fa92` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:46.049881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c04aac52296b46649c8e344aa9e0fa92)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:46.049881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:46.168744 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c04aac52296b46649c8e344aa9e0fa92', 'request_id': 'req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:46.179511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c04aac52296b46649c8e344aa9e0fa92', 'request_id': 'req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:46.187033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c04aac52296b46649c8e344aa9e0fa92', 'request_id': 'req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:46.255297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7 admin admin] The token cache is being invalidate because project c04aac52296b46649c8e344aa9e0fa92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:24:46.256019 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:46.256719 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:46.258603 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1085/2147] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:24:46 2022] DELETE /identity/v3/projects/c04aac52296b46649c8e344aa9e0fa92 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:46.263051 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05cb434-7ae5-4cdf-89d9-91b834b0d01e service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:46.274016 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05cb434-7ae5-4cdf-89d9-91b834b0d01e service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:46.369778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1985d0b-3609-4bb0-9a8b-f19c2701f6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:46.370752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1985d0b-3609-4bb0-9a8b-f19c2701f6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:46.371086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1985d0b-3609-4bb0-9a8b-f19c2701f6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:46.371374 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1985d0b-3609-4bb0-9a8b-f19c2701f6ce admin admin] PATH_INFO: `/v3/users/1fcef1152767492a9a2de847eab60d7b` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:46.373713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1985d0b-3609-4bb0-9a8b-f19c2701f6ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fcef1152767492a9a2de847eab60d7b)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:46.375945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1985d0b-3609-4bb0-9a8b-f19c2701f6ce admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:46.587687 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-d1985d0b-3609-4bb0-9a8b-f19c2701f6ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fcef1152767492a9a2de847eab60d7b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:46.592630 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:46.680186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-d1985d0b-3609-4bb0-9a8b-f19c2701f6ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fcef1152767492a9a2de847eab60d7b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:46.681265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-d1985d0b-3609-4bb0-9a8b-f19c2701f6ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fcef1152767492a9a2de847eab60d7b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:46.681996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-d1985d0b-3609-4bb0-9a8b-f19c2701f6ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fcef1152767492a9a2de847eab60d7b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:46.698876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-d1985d0b-3609-4bb0-9a8b-f19c2701f6ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fcef1152767492a9a2de847eab60d7b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:46.717653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1063/2148] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:24:46 2022] DELETE /identity/v3/users/1fcef1152767492a9a2de847eab60d7b => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:47.248187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:47.250585 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:47.270288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b50ef-abd3-4f11-b51f-4ab6ce1dc7b7 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:47.298366 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:47.337062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:47.337944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:47.338226 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:47.344106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13 admin admin] PATH_INFO: `/v3/projects/c8dff9e711094ebbb967b197f525cedd` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:47.344106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8dff9e711094ebbb967b197f525cedd)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:47.344106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:47.449364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8dff9e711094ebbb967b197f525cedd', 'request_id': 'req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:47.456557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8dff9e711094ebbb967b197f525cedd', 'request_id': 'req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:47.463332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8dff9e711094ebbb967b197f525cedd', 'request_id': 'req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:47.600447 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13 admin admin] The token cache is being invalidate because project c8dff9e711094ebbb967b197f525cedd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:24:47.600815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:47.601528 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:47.603175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1086/2149] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:24:47 2022] DELETE /identity/v3/projects/c8dff9e711094ebbb967b197f525cedd => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:47.607502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1985d0b-3609-4bb0-9a8b-f19c2701f6ce admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:47.609120 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d1985d0b-3609-4bb0-9a8b-f19c2701f6ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:47.624114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1985d0b-3609-4bb0-9a8b-f19c2701f6ce admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:47.693048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc92c4-f066-49f7-9c27-e7f715e24dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:47.693951 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc92c4-f066-49f7-9c27-e7f715e24dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:47.694236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc92c4-f066-49f7-9c27-e7f715e24dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:47.696547 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc92c4-f066-49f7-9c27-e7f715e24dd5 admin admin] PATH_INFO: `/v3/users/ab7adf70fce7421c96b679f94783f9d5` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:47.696547 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc92c4-f066-49f7-9c27-e7f715e24dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab7adf70fce7421c96b679f94783f9d5)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:47.698581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc92c4-f066-49f7-9c27-e7f715e24dd5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:47.941726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-53fc92c4-f066-49f7-9c27-e7f715e24dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab7adf70fce7421c96b679f94783f9d5'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:48.023189 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-53fc92c4-f066-49f7-9c27-e7f715e24dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab7adf70fce7421c96b679f94783f9d5'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:48.023965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-53fc92c4-f066-49f7-9c27-e7f715e24dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab7adf70fce7421c96b679f94783f9d5'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:48.024603 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-53fc92c4-f066-49f7-9c27-e7f715e24dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab7adf70fce7421c96b679f94783f9d5'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:48.041126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-53fc92c4-f066-49f7-9c27-e7f715e24dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab7adf70fce7421c96b679f94783f9d5'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:48.054067 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1064/2150] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:24:47 2022] DELETE /identity/v3/users/ab7adf70fce7421c96b679f94783f9d5 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:48.490778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:48.492281 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:48.510108 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a4ad7-030e-4f5e-8ebc-6bc4ca25cd13 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:48.574528 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37117726-804b-4f31-bbca-8ae2310482fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:48.575395 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37117726-804b-4f31-bbca-8ae2310482fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:48.575678 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37117726-804b-4f31-bbca-8ae2310482fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:48.576014 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37117726-804b-4f31-bbca-8ae2310482fb admin admin] PATH_INFO: `/v3/projects/6c18cb936b034993b652db0f15dc729b` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:48.577232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37117726-804b-4f31-bbca-8ae2310482fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c18cb936b034993b652db0f15dc729b)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:48.579169 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37117726-804b-4f31-bbca-8ae2310482fb admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:48.723795 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-37117726-804b-4f31-bbca-8ae2310482fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c18cb936b034993b652db0f15dc729b', 'request_id': 'req-37117726-804b-4f31-bbca-8ae2310482fb'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:48.729994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-37117726-804b-4f31-bbca-8ae2310482fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c18cb936b034993b652db0f15dc729b', 'request_id': 'req-37117726-804b-4f31-bbca-8ae2310482fb'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:48.734773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:48.737451 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-37117726-804b-4f31-bbca-8ae2310482fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c18cb936b034993b652db0f15dc729b', 'request_id': 'req-37117726-804b-4f31-bbca-8ae2310482fb'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:48.840880 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-37117726-804b-4f31-bbca-8ae2310482fb admin admin] The token cache is being invalidate because project 6c18cb936b034993b652db0f15dc729b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:24:48.841126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-37117726-804b-4f31-bbca-8ae2310482fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:48.842537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-37117726-804b-4f31-bbca-8ae2310482fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:24:48.844524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1087/2151] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:24:48 2022] DELETE /identity/v3/projects/6c18cb936b034993b652db0f15dc729b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:48.851846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475f1c7-9c1c-4ee2-ba74-f0db996d3ae5 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:48.852392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475f1c7-9c1c-4ee2-ba74-f0db996d3ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:48.852482 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475f1c7-9c1c-4ee2-ba74-f0db996d3ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:48.874933 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-1475f1c7-9c1c-4ee2-ba74-f0db996d3ae5 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:48.876260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1475f1c7-9c1c-4ee2-ba74-f0db996d3ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:48.930346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:48.952359 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1065/2152] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:48.961377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f733d-2a27-4e81-8992-5094c7178c6b None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:48.961667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f733d-2a27-4e81-8992-5094c7178c6b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:48.961945 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f733d-2a27-4e81-8992-5094c7178c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:48.986096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-a16f733d-2a27-4e81-8992-5094c7178c6b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:48.987320 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-a16f733d-2a27-4e81-8992-5094c7178c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:48.996338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1088/2153] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:49.000339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475f1c7-9c1c-4ee2-ba74-f0db996d3ae5 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:49.014821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475f1c7-9c1c-4ee2-ba74-f0db996d3ae5 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:49.021447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f86c7-7ef9-4ca8-bbc7-f7e71c3356e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:49.022415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f86c7-7ef9-4ca8-bbc7-f7e71c3356e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:49.022708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f86c7-7ef9-4ca8-bbc7-f7e71c3356e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:49.022992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f86c7-7ef9-4ca8-bbc7-f7e71c3356e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:49.023778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f86c7-7ef9-4ca8-bbc7-f7e71c3356e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:24:49.024086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f86c7-7ef9-4ca8-bbc7-f7e71c3356e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:49.026317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f86c7-7ef9-4ca8-bbc7-f7e71c3356e3 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:49.042425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1066/2154] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:24:49.049544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f733d-2a27-4e81-8992-5094c7178c6b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:49.059350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f733d-2a27-4e81-8992-5094c7178c6b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:49.064862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5f68e-12f0-4ee7-9f14-4fab8574606c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:49.065566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5f68e-12f0-4ee7-9f14-4fab8574606c admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:49.065855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5f68e-12f0-4ee7-9f14-4fab8574606c admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:49.066136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5f68e-12f0-4ee7-9f14-4fab8574606c admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:49.066834 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5f68e-12f0-4ee7-9f14-4fab8574606c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:49.068838 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5f68e-12f0-4ee7-9f14-4fab8574606c admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:49.163494 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1089/2155] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:24:49 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:24:49.168093 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f86c7-7ef9-4ca8-bbc7-f7e71c3356e3 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:49.179859 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f86c7-7ef9-4ca8-bbc7-f7e71c3356e3 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:49.244783 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2fa0d-ecfb-4fba-8029-0ddf2fc4de66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:49.245485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2fa0d-ecfb-4fba-8029-0ddf2fc4de66 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:49.245784 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2fa0d-ecfb-4fba-8029-0ddf2fc4de66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:49.246061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2fa0d-ecfb-4fba-8029-0ddf2fc4de66 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:49.250579 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2fa0d-ecfb-4fba-8029-0ddf2fc4de66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:49.252081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2fa0d-ecfb-4fba-8029-0ddf2fc4de66 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:49.378210 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1067/2156] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:24:49 2022] POST /identity/v3/users => generated 291 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:24:49.380425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5f68e-12f0-4ee7-9f14-4fab8574606c admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:49.391376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5f68e-12f0-4ee7-9f14-4fab8574606c admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:49.397010 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b51dd4-340f-4ba2-80a9-0456f89f71f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:49.397133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b51dd4-340f-4ba2-80a9-0456f89f71f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:49.397465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b51dd4-340f-4ba2-80a9-0456f89f71f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:49.397700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b51dd4-340f-4ba2-80a9-0456f89f71f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:49.398445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b51dd4-340f-4ba2-80a9-0456f89f71f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:49.400130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b51dd4-340f-4ba2-80a9-0456f89f71f8 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:49.420483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1090/2157] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:24:49.422081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2fa0d-ecfb-4fba-8029-0ddf2fc4de66 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:49.442976 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2fa0d-ecfb-4fba-8029-0ddf2fc4de66 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:49.448215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea770f7-0d0c-4038-bc5f-c4f9dcc970c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:49.449077 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea770f7-0d0c-4038-bc5f-c4f9dcc970c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:49.449363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea770f7-0d0c-4038-bc5f-c4f9dcc970c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:49.449641 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea770f7-0d0c-4038-bc5f-c4f9dcc970c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:49.450375 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea770f7-0d0c-4038-bc5f-c4f9dcc970c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:49.452117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea770f7-0d0c-4038-bc5f-c4f9dcc970c1 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:49.464475 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1068/2158] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:24:49.470435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b51dd4-340f-4ba2-80a9-0456f89f71f8 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:49.486738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b51dd4-340f-4ba2-80a9-0456f89f71f8 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:49.490492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d509ae-d61b-40ca-a8d2-e603b0181fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:49.491459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d509ae-d61b-40ca-a8d2-e603b0181fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:49.491966 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d509ae-d61b-40ca-a8d2-e603b0181fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:49.492063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d509ae-d61b-40ca-a8d2-e603b0181fe5 admin admin] PATH_INFO: `/v3/projects/40cd185858444d8aaf78e72a740b624f/users/80a318aa610e449fb6e480372ee61db8/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:49.513981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d509ae-d61b-40ca-a8d2-e603b0181fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cd185858444d8aaf78e72a740b624f, user_id=80a318aa610e449fb6e480372ee61db8, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:49.518332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d509ae-d61b-40ca-a8d2-e603b0181fe5 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:49.677618 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1091/2159] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:24:49 2022] PUT /identity/v3/projects/40cd185858444d8aaf78e72a740b624f/users/80a318aa610e449fb6e480372ee61db8/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:24:49.682792 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dd58a-dae7-46db-8b60-e52bcbb12237 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:49.683084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dd58a-dae7-46db-8b60-e52bcbb12237 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:49.683365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775dd58a-dae7-46db-8b60-e52bcbb12237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:49.717801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-775dd58a-dae7-46db-8b60-e52bcbb12237 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:49.719027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-775dd58a-dae7-46db-8b60-e52bcbb12237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:49.782607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1069/2160] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:49.792288 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d509ae-d61b-40ca-a8d2-e603b0181fe5 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:49.794075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-71d509ae-d61b-40ca-a8d2-e603b0181fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:49.863205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d509ae-d61b-40ca-a8d2-e603b0181fe5 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:49.866860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b0c98-64eb-4841-b106-dec69bb3d051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:49.867507 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b0c98-64eb-4841-b106-dec69bb3d051 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:49.867787 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b0c98-64eb-4841-b106-dec69bb3d051 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:49.868059 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b0c98-64eb-4841-b106-dec69bb3d051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:49.880540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b0c98-64eb-4841-b106-dec69bb3d051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:49.882065 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b0c98-64eb-4841-b106-dec69bb3d051 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:49.893254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1092/2161] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:51.237280 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff29a1-afa8-4d3a-9c9e-7d4c81d7dc46 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:51.237858 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff29a1-afa8-4d3a-9c9e-7d4c81d7dc46 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:51.238048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff29a1-afa8-4d3a-9c9e-7d4c81d7dc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:51.271913 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b9ff29a1-afa8-4d3a-9c9e-7d4c81d7dc46 None None] MFA Rules not processed for user `80a318aa610e449fb6e480372ee61db8`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:51.273167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b9ff29a1-afa8-4d3a-9c9e-7d4c81d7dc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:51.328054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:51.351062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1070/2162] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:51.361855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b0c98-64eb-4841-b106-dec69bb3d051 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:51.363219 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ff1b0c98-64eb-4841-b106-dec69bb3d051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:51.422488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b0c98-64eb-4841-b106-dec69bb3d051 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:51.426023 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537960a-46c9-4aa5-bb5c-53c2ba922453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:51.426670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537960a-46c9-4aa5-bb5c-53c2ba922453 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:51.426953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537960a-46c9-4aa5-bb5c-53c2ba922453 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:51.427230 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537960a-46c9-4aa5-bb5c-53c2ba922453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:51.435679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537960a-46c9-4aa5-bb5c-53c2ba922453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:51.437215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537960a-46c9-4aa5-bb5c-53c2ba922453 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:51.446127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1093/2163] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:24:53.047200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038938a5-c3f8-42ec-b38f-630f63c735a4 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:53.047756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038938a5-c3f8-42ec-b38f-630f63c735a4 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:53.047848 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038938a5-c3f8-42ec-b38f-630f63c735a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:53.113763 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-038938a5-c3f8-42ec-b38f-630f63c735a4 None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:53.114990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-038938a5-c3f8-42ec-b38f-630f63c735a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:53.184476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1071/2164] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jun 15 10:24:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:53.234919 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537960a-46c9-4aa5-bb5c-53c2ba922453 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:53.236332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-8537960a-46c9-4aa5-bb5c-53c2ba922453 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:53.270824 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:24:53.332552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537960a-46c9-4aa5-bb5c-53c2ba922453 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:53.337739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82364912-c35c-416a-96e7-3df4cc6686f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:53.338398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82364912-c35c-416a-96e7-3df4cc6686f3 service swift] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:53.338725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82364912-c35c-416a-96e7-3df4cc6686f3 service swift] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:53.339003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82364912-c35c-416a-96e7-3df4cc6686f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:53.348035 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82364912-c35c-416a-96e7-3df4cc6686f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:53.349727 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82364912-c35c-416a-96e7-3df4cc6686f3 service swift] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:53.358216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1094/2165] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 10:24:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:24:54.173097 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e468a9-d921-476c-8e97-26b321f688a8 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:54.173664 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e468a9-d921-476c-8e97-26b321f688a8 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:54.173766 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e468a9-d921-476c-8e97-26b321f688a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:54.195312 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-c7e468a9-d921-476c-8e97-26b321f688a8 None None] MFA Rules not processed for user `80a318aa610e449fb6e480372ee61db8`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:24:54.196825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c7e468a9-d921-476c-8e97-26b321f688a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:54.211953 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1072/2166] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:24:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:24:57.068394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82364912-c35c-416a-96e7-3df4cc6686f3 service swift] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:24:57.070115 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-82364912-c35c-416a-96e7-3df4cc6686f3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:57.169745 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82364912-c35c-416a-96e7-3df4cc6686f3 service swift] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:24:57.175626 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7a8f2-4b49-4d72-9bbe-b0f12a18c035 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:24:57.176201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7a8f2-4b49-4d72-9bbe-b0f12a18c035 service placement] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:24:57.176485 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7a8f2-4b49-4d72-9bbe-b0f12a18c035 service placement] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:24:57.176762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7a8f2-4b49-4d72-9bbe-b0f12a18c035 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:24:57.178610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b2d7a8f2-4b49-4d72-9bbe-b0f12a18c035 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:24:57.191355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7a8f2-4b49-4d72-9bbe-b0f12a18c035 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:24:57.193082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7a8f2-4b49-4d72-9bbe-b0f12a18c035 service placement] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:24:57.204520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1095/2167] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:24:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:04.188425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=54 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:07.199704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=69 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:13.994275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d02bc-e99f-4ed8-9ecb-35b66d001e51 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:13.995023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d02bc-e99f-4ed8-9ecb-35b66d001e51 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:13.995454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d02bc-e99f-4ed8-9ecb-35b66d001e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:14.021237 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-cc2d02bc-e99f-4ed8-9ecb-35b66d001e51 None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:14.022670 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-cc2d02bc-e99f-4ed8-9ecb-35b66d001e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:14.031143 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1073/2168] 10.222.0.60 () {58 vars in 967 bytes} [Wed Jun 15 10:25:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:14.047448 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7a8f2-4b49-4d72-9bbe-b0f12a18c035 service placement] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:14.059163 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7a8f2-4b49-4d72-9bbe-b0f12a18c035 service placement] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:14.065762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22741d-266f-4542-8028-0a6e67ed6a10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:14.066528 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22741d-266f-4542-8028-0a6e67ed6a10 service swift] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:14.066770 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22741d-266f-4542-8028-0a6e67ed6a10 service swift] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:14.067371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22741d-266f-4542-8028-0a6e67ed6a10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:14.076491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22741d-266f-4542-8028-0a6e67ed6a10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:14.078811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22741d-266f-4542-8028-0a6e67ed6a10 service swift] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:14.087109 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1096/2169] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 10:25:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:25:17.282891 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d02bc-e99f-4ed8-9ecb-35b66d001e51 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:17.284609 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-cc2d02bc-e99f-4ed8-9ecb-35b66d001e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:17.302367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d02bc-e99f-4ed8-9ecb-35b66d001e51 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:17.309139 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0c346-15e6-47ac-923f-a3c2abfc42f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:17.310066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0c346-15e6-47ac-923f-a3c2abfc42f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:17.310351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0c346-15e6-47ac-923f-a3c2abfc42f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:17.310628 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0c346-15e6-47ac-923f-a3c2abfc42f5 admin admin] PATH_INFO: `/v3/users/1d4be529ee5e4765ada8eaee33e21c64` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:17.312047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0c346-15e6-47ac-923f-a3c2abfc42f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d4be529ee5e4765ada8eaee33e21c64)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:17.314331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0c346-15e6-47ac-923f-a3c2abfc42f5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:17.558057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b0c0c346-15e6-47ac-923f-a3c2abfc42f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d4be529ee5e4765ada8eaee33e21c64'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:17.662966 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b0c0c346-15e6-47ac-923f-a3c2abfc42f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d4be529ee5e4765ada8eaee33e21c64'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:17.663597 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b0c0c346-15e6-47ac-923f-a3c2abfc42f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d4be529ee5e4765ada8eaee33e21c64'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:17.664204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b0c0c346-15e6-47ac-923f-a3c2abfc42f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d4be529ee5e4765ada8eaee33e21c64'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:17.676691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b0c0c346-15e6-47ac-923f-a3c2abfc42f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d4be529ee5e4765ada8eaee33e21c64'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:17.689513 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1074/2170] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:25:17 2022] DELETE /identity/v3/users/1d4be529ee5e4765ada8eaee33e21c64 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:25:17.985677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22741d-266f-4542-8028-0a6e67ed6a10 service swift] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:18.010269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22741d-266f-4542-8028-0a6e67ed6a10 service swift] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:18.070699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e21236-ffa2-42cf-87b3-29c3ed6d4809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:18.071537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e21236-ffa2-42cf-87b3-29c3ed6d4809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:18.071872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e21236-ffa2-42cf-87b3-29c3ed6d4809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:18.072155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e21236-ffa2-42cf-87b3-29c3ed6d4809 admin admin] PATH_INFO: `/v3/projects/e73242c46698496ba8b80ce681417938` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:18.073746 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e21236-ffa2-42cf-87b3-29c3ed6d4809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e73242c46698496ba8b80ce681417938)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:18.075660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e21236-ffa2-42cf-87b3-29c3ed6d4809 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:18.210135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-58e21236-ffa2-42cf-87b3-29c3ed6d4809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e73242c46698496ba8b80ce681417938', 'request_id': 'req-58e21236-ffa2-42cf-87b3-29c3ed6d4809'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:18.217155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-58e21236-ffa2-42cf-87b3-29c3ed6d4809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e73242c46698496ba8b80ce681417938', 'request_id': 'req-58e21236-ffa2-42cf-87b3-29c3ed6d4809'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:18.223237 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-58e21236-ffa2-42cf-87b3-29c3ed6d4809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e73242c46698496ba8b80ce681417938', 'request_id': 'req-58e21236-ffa2-42cf-87b3-29c3ed6d4809'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:18.310173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-58e21236-ffa2-42cf-87b3-29c3ed6d4809 admin admin] The token cache is being invalidate because project e73242c46698496ba8b80ce681417938 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:25:18.311846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-58e21236-ffa2-42cf-87b3-29c3ed6d4809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:18.311846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-58e21236-ffa2-42cf-87b3-29c3ed6d4809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:18.314016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1097/2171] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:25:17 2022] DELETE /identity/v3/projects/e73242c46698496ba8b80ce681417938 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:25:18.318802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f673cea-18f9-4a91-b1ad-c191b4446fac None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:18.319734 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f673cea-18f9-4a91-b1ad-c191b4446fac None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:18.320192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f673cea-18f9-4a91-b1ad-c191b4446fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:18.342723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-3f673cea-18f9-4a91-b1ad-c191b4446fac None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:18.343969 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3f673cea-18f9-4a91-b1ad-c191b4446fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:18.412542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1075/2172] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:18.420702 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240cc3bb-9aa3-4023-adbd-58254aff4f27 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:18.425585 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240cc3bb-9aa3-4023-adbd-58254aff4f27 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:18.425585 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240cc3bb-9aa3-4023-adbd-58254aff4f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:18.442979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-240cc3bb-9aa3-4023-adbd-58254aff4f27 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:18.444223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-240cc3bb-9aa3-4023-adbd-58254aff4f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:18.456490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1098/2173] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:18.457846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f673cea-18f9-4a91-b1ad-c191b4446fac None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:18.467829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f673cea-18f9-4a91-b1ad-c191b4446fac None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:18.476141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787e359-6377-4016-9b19-4457156a3bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:18.476826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787e359-6377-4016-9b19-4457156a3bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:18.477159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787e359-6377-4016-9b19-4457156a3bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:18.477455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787e359-6377-4016-9b19-4457156a3bb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:18.478220 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787e359-6377-4016-9b19-4457156a3bb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:25:18.478538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787e359-6377-4016-9b19-4457156a3bb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:18.480142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787e359-6377-4016-9b19-4457156a3bb7 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:18.497005 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1076/2174] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:25:18.498418 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240cc3bb-9aa3-4023-adbd-58254aff4f27 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:18.509147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240cc3bb-9aa3-4023-adbd-58254aff4f27 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:18.514728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc4d90-1c40-47a7-bbb5-a121b4f54dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:18.517715 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc4d90-1c40-47a7-bbb5-a121b4f54dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:18.517715 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc4d90-1c40-47a7-bbb5-a121b4f54dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:18.517715 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc4d90-1c40-47a7-bbb5-a121b4f54dd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:18.517715 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc4d90-1c40-47a7-bbb5-a121b4f54dd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:18.519430 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc4d90-1c40-47a7-bbb5-a121b4f54dd9 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:18.605386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787e359-6377-4016-9b19-4457156a3bb7 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:18.606544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1099/2175] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:25:18 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:25:18.621175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787e359-6377-4016-9b19-4457156a3bb7 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:18.699594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:18.711212 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3da29-8928-48c2-bda7-3df59363f755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:18.712044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3da29-8928-48c2-bda7-3df59363f755 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:18.712335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3da29-8928-48c2-bda7-3df59363f755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:18.712613 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3da29-8928-48c2-bda7-3df59363f755 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:18.713351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3da29-8928-48c2-bda7-3df59363f755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:18.715421 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3da29-8928-48c2-bda7-3df59363f755 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:18.784989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1077/2176] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:25:18 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:25:18.789403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc4d90-1c40-47a7-bbb5-a121b4f54dd9 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:18.801335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc4d90-1c40-47a7-bbb5-a121b4f54dd9 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:18.806187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e76d9-a830-40d5-8f04-04c3873fa483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:18.810072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e76d9-a830-40d5-8f04-04c3873fa483 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:18.810072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e76d9-a830-40d5-8f04-04c3873fa483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:18.810072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e76d9-a830-40d5-8f04-04c3873fa483 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:18.810072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e76d9-a830-40d5-8f04-04c3873fa483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:18.810072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e76d9-a830-40d5-8f04-04c3873fa483 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:18.820998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1100/2177] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:25:18.825443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3da29-8928-48c2-bda7-3df59363f755 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:18.844631 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3da29-8928-48c2-bda7-3df59363f755 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:18.844631 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2ff58-760b-4dc9-9350-696ae01bccc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:18.844631 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2ff58-760b-4dc9-9350-696ae01bccc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:18.844631 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2ff58-760b-4dc9-9350-696ae01bccc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:18.845178 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2ff58-760b-4dc9-9350-696ae01bccc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:18.845178 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2ff58-760b-4dc9-9350-696ae01bccc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:18.845178 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2ff58-760b-4dc9-9350-696ae01bccc8 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:18.860223 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1078/2178] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:25:18.861328 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e76d9-a830-40d5-8f04-04c3873fa483 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:18.871733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e76d9-a830-40d5-8f04-04c3873fa483 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:18.877296 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b612-55ba-4e73-ba1a-28f353749401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:18.878220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b612-55ba-4e73-ba1a-28f353749401 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:18.878504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b612-55ba-4e73-ba1a-28f353749401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:18.879204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b612-55ba-4e73-ba1a-28f353749401 admin admin] PATH_INFO: `/v3/projects/2b7f39e382d94a9a9636655caafda62c/users/204bd8201166497e8b79125d5d1771d0/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:18.898093 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3b612-55ba-4e73-ba1a-28f353749401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b7f39e382d94a9a9636655caafda62c, user_id=204bd8201166497e8b79125d5d1771d0, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:18.901040 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3b612-55ba-4e73-ba1a-28f353749401 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:18.978296 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1101/2179] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:25:18 2022] PUT /identity/v3/projects/2b7f39e382d94a9a9636655caafda62c/users/204bd8201166497e8b79125d5d1771d0/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:25:18.988564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ff6e8-b768-49e8-bb52-58b70bfb0d7a None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:18.988861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ff6e8-b768-49e8-bb52-58b70bfb0d7a None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:18.989194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ff6e8-b768-49e8-bb52-58b70bfb0d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:19.025721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-c67ff6e8-b768-49e8-bb52-58b70bfb0d7a None None] MFA Rules not processed for user `204bd8201166497e8b79125d5d1771d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:19.027378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c67ff6e8-b768-49e8-bb52-58b70bfb0d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:19.152041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1079/2180] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:25:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:19.170684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b612-55ba-4e73-ba1a-28f353749401 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:19.170684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-cac3b612-55ba-4e73-ba1a-28f353749401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:19.185311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:19.254744 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b612-55ba-4e73-ba1a-28f353749401 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:19.259039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264f427-d67a-4e63-9739-58083d7520a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:19.259912 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264f427-d67a-4e63-9739-58083d7520a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:19.260354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264f427-d67a-4e63-9739-58083d7520a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:19.261318 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264f427-d67a-4e63-9739-58083d7520a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:19.278852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264f427-d67a-4e63-9739-58083d7520a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:19.280341 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264f427-d67a-4e63-9739-58083d7520a9 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:19.304678 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1102/2181] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:26.919612 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ff6e8-b768-49e8-bb52-58b70bfb0d7a None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:27.001292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ff6e8-b768-49e8-bb52-58b70bfb0d7a None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:27.006850 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de3ad5-0ac3-43d8-868e-383c41ed189a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:27.007702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de3ad5-0ac3-43d8-868e-383c41ed189a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:27.007989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de3ad5-0ac3-43d8-868e-383c41ed189a admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:27.008265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de3ad5-0ac3-43d8-868e-383c41ed189a admin admin] PATH_INFO: `/v3/users/204bd8201166497e8b79125d5d1771d0` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:27.009844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de3ad5-0ac3-43d8-868e-383c41ed189a admin admin] RBAC: Authorizing `identity:delete_user(user_id=204bd8201166497e8b79125d5d1771d0)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:27.012089 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de3ad5-0ac3-43d8-868e-383c41ed189a admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:27.216842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-42de3ad5-0ac3-43d8-868e-383c41ed189a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '204bd8201166497e8b79125d5d1771d0'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:27.291444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-42de3ad5-0ac3-43d8-868e-383c41ed189a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '204bd8201166497e8b79125d5d1771d0'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:27.292095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-42de3ad5-0ac3-43d8-868e-383c41ed189a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '204bd8201166497e8b79125d5d1771d0'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:27.292724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-42de3ad5-0ac3-43d8-868e-383c41ed189a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '204bd8201166497e8b79125d5d1771d0'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:27.301567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:27.310621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-42de3ad5-0ac3-43d8-868e-383c41ed189a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '204bd8201166497e8b79125d5d1771d0'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:27.324216 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1080/2182] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:25:26 2022] DELETE /identity/v3/users/204bd8201166497e8b79125d5d1771d0 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:25:27.328615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f0781-d86f-49d1-ba20-28095e757e71 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:27.329733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f0781-d86f-49d1-ba20-28095e757e71 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:27.329904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f0781-d86f-49d1-ba20-28095e757e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:27.348207 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-5f3f0781-d86f-49d1-ba20-28095e757e71 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:27.349466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5f3f0781-d86f-49d1-ba20-28095e757e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:27.417965 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1103/2183] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:27.428162 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de3ad5-0ac3-43d8-868e-383c41ed189a admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:27.429684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-42de3ad5-0ac3-43d8-868e-383c41ed189a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:27.511335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de3ad5-0ac3-43d8-868e-383c41ed189a admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:27.515265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953c045-eb7f-438c-abae-4bf3dedc91d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:27.515988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953c045-eb7f-438c-abae-4bf3dedc91d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:27.516270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953c045-eb7f-438c-abae-4bf3dedc91d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:27.517827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953c045-eb7f-438c-abae-4bf3dedc91d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:27.531447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953c045-eb7f-438c-abae-4bf3dedc91d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:27.531447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953c045-eb7f-438c-abae-4bf3dedc91d5 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:27.541666 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1081/2184] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:28.265699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f0781-d86f-49d1-ba20-28095e757e71 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:28.267555 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5f3f0781-d86f-49d1-ba20-28095e757e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:28.290677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f0781-d86f-49d1-ba20-28095e757e71 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:28.297381 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b210a-bdf9-4651-903d-50b7eab73116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:28.298954 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b210a-bdf9-4651-903d-50b7eab73116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:28.299315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b210a-bdf9-4651-903d-50b7eab73116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:28.299594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b210a-bdf9-4651-903d-50b7eab73116 admin admin] PATH_INFO: `/v3/projects/2b7f39e382d94a9a9636655caafda62c` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:28.301496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b210a-bdf9-4651-903d-50b7eab73116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b7f39e382d94a9a9636655caafda62c)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:28.308282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b210a-bdf9-4651-903d-50b7eab73116 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:28.403506 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ec9b210a-bdf9-4651-903d-50b7eab73116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b7f39e382d94a9a9636655caafda62c', 'request_id': 'req-ec9b210a-bdf9-4651-903d-50b7eab73116'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:28.412058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ec9b210a-bdf9-4651-903d-50b7eab73116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b7f39e382d94a9a9636655caafda62c', 'request_id': 'req-ec9b210a-bdf9-4651-903d-50b7eab73116'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:28.425456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ec9b210a-bdf9-4651-903d-50b7eab73116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b7f39e382d94a9a9636655caafda62c', 'request_id': 'req-ec9b210a-bdf9-4651-903d-50b7eab73116'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:28.521978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ec9b210a-bdf9-4651-903d-50b7eab73116 admin admin] The token cache is being invalidate because project 2b7f39e382d94a9a9636655caafda62c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:25:28.521978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ec9b210a-bdf9-4651-903d-50b7eab73116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:28.522282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ec9b210a-bdf9-4651-903d-50b7eab73116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:28.524614 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1104/2185] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:25:28 2022] DELETE /identity/v3/projects/2b7f39e382d94a9a9636655caafda62c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:25:28.530505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dea73b-7fbc-41a5-97c8-803f908b154e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:28.531564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dea73b-7fbc-41a5-97c8-803f908b154e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:28.531847 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dea73b-7fbc-41a5-97c8-803f908b154e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:28.558348 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-97dea73b-7fbc-41a5-97c8-803f908b154e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:28.559792 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-97dea73b-7fbc-41a5-97c8-803f908b154e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:28.633929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1082/2186] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:28.642233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17e872-e1b7-4940-8543-3cf160f68d0c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:28.642725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17e872-e1b7-4940-8543-3cf160f68d0c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:28.643103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17e872-e1b7-4940-8543-3cf160f68d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:28.663061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-3e17e872-e1b7-4940-8543-3cf160f68d0c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:28.664416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-3e17e872-e1b7-4940-8543-3cf160f68d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:28.673901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1105/2187] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:28.678141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dea73b-7fbc-41a5-97c8-803f908b154e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:28.688600 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dea73b-7fbc-41a5-97c8-803f908b154e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:28.693584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dba3b2-29f0-476b-a5ad-ff2a47b8ac5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:28.694249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dba3b2-29f0-476b-a5ad-ff2a47b8ac5c admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:28.694532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dba3b2-29f0-476b-a5ad-ff2a47b8ac5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:28.694804 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dba3b2-29f0-476b-a5ad-ff2a47b8ac5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:28.695543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dba3b2-29f0-476b-a5ad-ff2a47b8ac5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:25:28.695883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dba3b2-29f0-476b-a5ad-ff2a47b8ac5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:28.697740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dba3b2-29f0-476b-a5ad-ff2a47b8ac5c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:28.712610 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1083/2188] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:25:28.716620 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17e872-e1b7-4940-8543-3cf160f68d0c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:28.727132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17e872-e1b7-4940-8543-3cf160f68d0c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:28.732247 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4d7a4-6b81-40d8-aae5-3942fc205578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:28.732926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4d7a4-6b81-40d8-aae5-3942fc205578 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:28.733244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4d7a4-6b81-40d8-aae5-3942fc205578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:28.733524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4d7a4-6b81-40d8-aae5-3942fc205578 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:28.734241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4d7a4-6b81-40d8-aae5-3942fc205578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:28.736467 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4d7a4-6b81-40d8-aae5-3942fc205578 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:28.821004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1106/2189] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:25:28 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:25:28.825453 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dba3b2-29f0-476b-a5ad-ff2a47b8ac5c admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:28.846328 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dba3b2-29f0-476b-a5ad-ff2a47b8ac5c admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:28.927613 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef768f-0a25-4b1a-a8ae-6e42efe5d774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:28.928520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef768f-0a25-4b1a-a8ae-6e42efe5d774 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:28.928868 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef768f-0a25-4b1a-a8ae-6e42efe5d774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:28.929172 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef768f-0a25-4b1a-a8ae-6e42efe5d774 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:28.930007 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ef768f-0a25-4b1a-a8ae-6e42efe5d774 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:28.933049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ef768f-0a25-4b1a-a8ae-6e42efe5d774 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:29.036163 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:29.053869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1084/2190] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:25:28 2022] POST /identity/v3/users => generated 290 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:25:29.056497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4d7a4-6b81-40d8-aae5-3942fc205578 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:29.083153 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4d7a4-6b81-40d8-aae5-3942fc205578 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:29.093946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187f66a-6e5f-40c4-8aa5-0d155046518d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:29.093946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187f66a-6e5f-40c4-8aa5-0d155046518d admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:29.093946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187f66a-6e5f-40c4-8aa5-0d155046518d admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:29.093946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187f66a-6e5f-40c4-8aa5-0d155046518d admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:29.097411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187f66a-6e5f-40c4-8aa5-0d155046518d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:29.099173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187f66a-6e5f-40c4-8aa5-0d155046518d admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:29.105334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:29.112372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1107/2191] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:25:29.116687 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef768f-0a25-4b1a-a8ae-6e42efe5d774 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:29.136036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef768f-0a25-4b1a-a8ae-6e42efe5d774 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:29.141067 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5159c3-b85b-4b23-8c00-b7ab609e7af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:29.142593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5159c3-b85b-4b23-8c00-b7ab609e7af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:29.143336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5159c3-b85b-4b23-8c00-b7ab609e7af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:29.143336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5159c3-b85b-4b23-8c00-b7ab609e7af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:29.143887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5159c3-b85b-4b23-8c00-b7ab609e7af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:29.149021 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5159c3-b85b-4b23-8c00-b7ab609e7af3 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:29.170446 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1085/2192] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:25:29.171913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187f66a-6e5f-40c4-8aa5-0d155046518d admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:29.184224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187f66a-6e5f-40c4-8aa5-0d155046518d admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:29.191511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef476d7-1d74-468e-af56-d423cc44c84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:29.193722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef476d7-1d74-468e-af56-d423cc44c84c admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:29.193722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef476d7-1d74-468e-af56-d423cc44c84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:29.194278 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef476d7-1d74-468e-af56-d423cc44c84c admin admin] PATH_INFO: `/v3/projects/dedc3b6e0d964cb9937f0a57894554b7/users/0ac3d9b2893b4f078c2a379696aaff34/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:29.217525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef476d7-1d74-468e-af56-d423cc44c84c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dedc3b6e0d964cb9937f0a57894554b7, user_id=0ac3d9b2893b4f078c2a379696aaff34, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:29.221532 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef476d7-1d74-468e-af56-d423cc44c84c admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:29.321270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1108/2193] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:25:29 2022] PUT /identity/v3/projects/dedc3b6e0d964cb9937f0a57894554b7/users/0ac3d9b2893b4f078c2a379696aaff34/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:25:29.324215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe286b10-9060-40d5-8fc3-09c82e700acb None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:29.324725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe286b10-9060-40d5-8fc3-09c82e700acb None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:29.325007 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe286b10-9060-40d5-8fc3-09c82e700acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:29.351833 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-fe286b10-9060-40d5-8fc3-09c82e700acb None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:29.352816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-fe286b10-9060-40d5-8fc3-09c82e700acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:29.460523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1086/2194] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:29.473189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef476d7-1d74-468e-af56-d423cc44c84c admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:29.475537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-bef476d7-1d74-468e-af56-d423cc44c84c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:29.614942 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef476d7-1d74-468e-af56-d423cc44c84c admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:29.620052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47c8f3-c3bf-4f26-abf2-e8f9f6b3ac9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:29.620640 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47c8f3-c3bf-4f26-abf2-e8f9f6b3ac9a service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:29.620924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47c8f3-c3bf-4f26-abf2-e8f9f6b3ac9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:29.621255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47c8f3-c3bf-4f26-abf2-e8f9f6b3ac9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:29.632833 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c47c8f3-c3bf-4f26-abf2-e8f9f6b3ac9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:29.634399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c47c8f3-c3bf-4f26-abf2-e8f9f6b3ac9a service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:29.656142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1109/2195] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:30.123364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe286b10-9060-40d5-8fc3-09c82e700acb None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:30.125473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-fe286b10-9060-40d5-8fc3-09c82e700acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:30.138778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe286b10-9060-40d5-8fc3-09c82e700acb None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:30.144319 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e9ff0-374f-4284-8a73-7f09cc208d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:30.145236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e9ff0-374f-4284-8a73-7f09cc208d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:30.145521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e9ff0-374f-4284-8a73-7f09cc208d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:30.145797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e9ff0-374f-4284-8a73-7f09cc208d20 admin admin] PATH_INFO: `/v3/users/0212b3d4ad654667ab2f896d63919242` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:30.147045 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e9ff0-374f-4284-8a73-7f09cc208d20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0212b3d4ad654667ab2f896d63919242)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:30.149057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75e9ff0-374f-4284-8a73-7f09cc208d20 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:30.345653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e75e9ff0-374f-4284-8a73-7f09cc208d20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0212b3d4ad654667ab2f896d63919242'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:30.411906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e75e9ff0-374f-4284-8a73-7f09cc208d20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0212b3d4ad654667ab2f896d63919242'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:30.412577 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e75e9ff0-374f-4284-8a73-7f09cc208d20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0212b3d4ad654667ab2f896d63919242'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:30.413288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e75e9ff0-374f-4284-8a73-7f09cc208d20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0212b3d4ad654667ab2f896d63919242'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:30.427303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e75e9ff0-374f-4284-8a73-7f09cc208d20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0212b3d4ad654667ab2f896d63919242'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:30.440837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1087/2196] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:25:30 2022] DELETE /identity/v3/users/0212b3d4ad654667ab2f896d63919242 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:25:30.445660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979b4a4-773f-4f70-aaab-165dd8ea25c3 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:30.446030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979b4a4-773f-4f70-aaab-165dd8ea25c3 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:30.446388 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979b4a4-773f-4f70-aaab-165dd8ea25c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:30.465717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-b979b4a4-773f-4f70-aaab-165dd8ea25c3 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:30.467062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b979b4a4-773f-4f70-aaab-165dd8ea25c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:30.530167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1110/2197] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:30.543481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e9ff0-374f-4284-8a73-7f09cc208d20 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:30.545944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e75e9ff0-374f-4284-8a73-7f09cc208d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:30.605879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e9ff0-374f-4284-8a73-7f09cc208d20 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:30.611255 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce8b4c-f3b4-4dfb-b9aa-60f6929a6116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:30.614458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce8b4c-f3b4-4dfb-b9aa-60f6929a6116 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:30.614458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce8b4c-f3b4-4dfb-b9aa-60f6929a6116 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:30.614458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce8b4c-f3b4-4dfb-b9aa-60f6929a6116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:30.624657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce8b4c-f3b4-4dfb-b9aa-60f6929a6116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:30.626226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce8b4c-f3b4-4dfb-b9aa-60f6929a6116 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:30.637579 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1088/2198] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:31.167989 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979b4a4-773f-4f70-aaab-165dd8ea25c3 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:31.169804 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b979b4a4-773f-4f70-aaab-165dd8ea25c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:31.187251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979b4a4-773f-4f70-aaab-165dd8ea25c3 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:31.192130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc61274-25c9-4bbb-aa17-55a0a4e1f0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:31.193104 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc61274-25c9-4bbb-aa17-55a0a4e1f0ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:31.193400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc61274-25c9-4bbb-aa17-55a0a4e1f0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:31.193652 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc61274-25c9-4bbb-aa17-55a0a4e1f0ff admin admin] PATH_INFO: `/v3/projects/75f8396469a743d7a9b34f228b53e3cd` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:31.194877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc61274-25c9-4bbb-aa17-55a0a4e1f0ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f8396469a743d7a9b34f228b53e3cd)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:31.196808 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc61274-25c9-4bbb-aa17-55a0a4e1f0ff admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:31.373443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-9cc61274-25c9-4bbb-aa17-55a0a4e1f0ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f8396469a743d7a9b34f228b53e3cd', 'request_id': 'req-9cc61274-25c9-4bbb-aa17-55a0a4e1f0ff'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:31.381574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-9cc61274-25c9-4bbb-aa17-55a0a4e1f0ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f8396469a743d7a9b34f228b53e3cd', 'request_id': 'req-9cc61274-25c9-4bbb-aa17-55a0a4e1f0ff'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:31.387943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-9cc61274-25c9-4bbb-aa17-55a0a4e1f0ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f8396469a743d7a9b34f228b53e3cd', 'request_id': 'req-9cc61274-25c9-4bbb-aa17-55a0a4e1f0ff'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:31.461032 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-9cc61274-25c9-4bbb-aa17-55a0a4e1f0ff admin admin] The token cache is being invalidate because project 75f8396469a743d7a9b34f228b53e3cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:25:31.461378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-9cc61274-25c9-4bbb-aa17-55a0a4e1f0ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:31.462058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-9cc61274-25c9-4bbb-aa17-55a0a4e1f0ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:25:31.463707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1111/2199] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:25:31 2022] DELETE /identity/v3/projects/75f8396469a743d7a9b34f228b53e3cd => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:25:31.469196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf7b4d-d5b5-4dd5-b649-4de84dc32e2f None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:31.469697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf7b4d-d5b5-4dd5-b649-4de84dc32e2f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:31.469805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf7b4d-d5b5-4dd5-b649-4de84dc32e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:31.478086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:31.489301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-81bf7b4d-d5b5-4dd5-b649-4de84dc32e2f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:31.490532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-81bf7b4d-d5b5-4dd5-b649-4de84dc32e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:31.567566 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1089/2200] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:31.575928 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce80e0a-b17d-4731-875c-6b68e26998e7 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:31.576482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce80e0a-b17d-4731-875c-6b68e26998e7 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:31.576577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce80e0a-b17d-4731-875c-6b68e26998e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:31.586825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:31.602480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-0ce80e0a-b17d-4731-875c-6b68e26998e7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:31.603717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0ce80e0a-b17d-4731-875c-6b68e26998e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:31.612506 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1112/2201] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:31.617609 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf7b4d-d5b5-4dd5-b649-4de84dc32e2f None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:31.628610 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf7b4d-d5b5-4dd5-b649-4de84dc32e2f None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:31.633694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1796ce-d761-4c73-ab98-b5935c719199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:31.634745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1796ce-d761-4c73-ab98-b5935c719199 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:31.635468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1796ce-d761-4c73-ab98-b5935c719199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:31.636328 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1796ce-d761-4c73-ab98-b5935c719199 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:31.637474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1796ce-d761-4c73-ab98-b5935c719199 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:25:31.637780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1796ce-d761-4c73-ab98-b5935c719199 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:31.639424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1796ce-d761-4c73-ab98-b5935c719199 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:31.654519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1090/2202] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 10:25:31.659648 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce80e0a-b17d-4731-875c-6b68e26998e7 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:31.669823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce80e0a-b17d-4731-875c-6b68e26998e7 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:31.675188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c7093-3a8a-47d7-95c5-748d135e1113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:31.675946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c7093-3a8a-47d7-95c5-748d135e1113 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:31.676299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c7093-3a8a-47d7-95c5-748d135e1113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:31.676640 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c7093-3a8a-47d7-95c5-748d135e1113 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:31.677439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c7093-3a8a-47d7-95c5-748d135e1113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:31.679528 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c7093-3a8a-47d7-95c5-748d135e1113 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:31.792568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1113/2203] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:25:31 2022] POST /identity/v3/projects => generated 363 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:25:31.797462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1796ce-d761-4c73-ab98-b5935c719199 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:31.810503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1796ce-d761-4c73-ab98-b5935c719199 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:31.931681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7df0b-327c-44ef-9637-744802b6ec81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:31.932514 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7df0b-327c-44ef-9637-744802b6ec81 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:31.932809 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7df0b-327c-44ef-9637-744802b6ec81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:31.933123 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7df0b-327c-44ef-9637-744802b6ec81 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:31.933843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7df0b-327c-44ef-9637-744802b6ec81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:31.935823 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7df0b-327c-44ef-9637-744802b6ec81 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:32.041858 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1091/2204] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:25:31 2022] POST /identity/v3/users => generated 282 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:25:32.049638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c7093-3a8a-47d7-95c5-748d135e1113 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:32.065110 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c7093-3a8a-47d7-95c5-748d135e1113 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:32.071164 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53629bac-a28d-45aa-b526-1cdec24ffb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:32.073250 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53629bac-a28d-45aa-b526-1cdec24ffb42 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:32.074196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53629bac-a28d-45aa-b526-1cdec24ffb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:32.074507 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53629bac-a28d-45aa-b526-1cdec24ffb42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:32.075479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53629bac-a28d-45aa-b526-1cdec24ffb42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:32.078032 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53629bac-a28d-45aa-b526-1cdec24ffb42 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:32.097603 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7df0b-327c-44ef-9637-744802b6ec81 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:32.117272 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1114/2205] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:25:32.128388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7df0b-327c-44ef-9637-744802b6ec81 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:32.146253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483cebb5-6e2e-494c-8679-315acec0bf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:32.148923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cebb5-6e2e-494c-8679-315acec0bf4a admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:32.149722 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cebb5-6e2e-494c-8679-315acec0bf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:32.150298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cebb5-6e2e-494c-8679-315acec0bf4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:32.152214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483cebb5-6e2e-494c-8679-315acec0bf4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:32.153908 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483cebb5-6e2e-494c-8679-315acec0bf4a admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:32.171084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1092/2206] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:25:32.172136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53629bac-a28d-45aa-b526-1cdec24ffb42 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:32.189740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53629bac-a28d-45aa-b526-1cdec24ffb42 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:32.196304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dfd9a-7dde-44f1-ac31-c1aa5ce0386a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:32.197403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dfd9a-7dde-44f1-ac31-c1aa5ce0386a admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:32.197683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dfd9a-7dde-44f1-ac31-c1aa5ce0386a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:32.198018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dfd9a-7dde-44f1-ac31-c1aa5ce0386a admin admin] PATH_INFO: `/v3/projects/85bf4851de0647b4a51b5b3699f4b206/users/3aac0bb570d24c069b7467be64085e2f/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:32.215796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3dfd9a-7dde-44f1-ac31-c1aa5ce0386a admin admin] RBAC: Authorizing `identity:create_grant(project_id=85bf4851de0647b4a51b5b3699f4b206, user_id=3aac0bb570d24c069b7467be64085e2f, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:32.218749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3dfd9a-7dde-44f1-ac31-c1aa5ce0386a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:32.300782 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1115/2207] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:25:32 2022] PUT /identity/v3/projects/85bf4851de0647b4a51b5b3699f4b206/users/3aac0bb570d24c069b7467be64085e2f/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:25:32.312611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e9558-e233-42e9-8a73-3c80ef2ba65c None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:32.312912 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e9558-e233-42e9-8a73-3c80ef2ba65c None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:32.313278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e9558-e233-42e9-8a73-3c80ef2ba65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:32.360512 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-561e9558-e233-42e9-8a73-3c80ef2ba65c None None] MFA Rules not processed for user `3aac0bb570d24c069b7467be64085e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:32.366905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-561e9558-e233-42e9-8a73-3c80ef2ba65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:32.470811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:32.479398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1093/2208] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:25:32 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:32.488267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dfd9a-7dde-44f1-ac31-c1aa5ce0386a admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:32.490123 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-af3dfd9a-7dde-44f1-ac31-c1aa5ce0386a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:32.597350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dfd9a-7dde-44f1-ac31-c1aa5ce0386a admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:32.601261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424739d0-2937-4ae4-a37a-89d534fca059 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:32.602217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424739d0-2937-4ae4-a37a-89d534fca059 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:32.602425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424739d0-2937-4ae4-a37a-89d534fca059 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:32.602862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424739d0-2937-4ae4-a37a-89d534fca059 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:32.615616 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424739d0-2937-4ae4-a37a-89d534fca059 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:32.617202 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424739d0-2937-4ae4-a37a-89d534fca059 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:32.627990 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1116/2209] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:32 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:35.926572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0e068-945f-4761-9e76-40ac7c96e5f5 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:35.926572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0e068-945f-4761-9e76-40ac7c96e5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:35.926572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0e068-945f-4761-9e76-40ac7c96e5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:35.971277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-9aa0e068-945f-4761-9e76-40ac7c96e5f5 None None] MFA Rules not processed for user `0ac3d9b2893b4f078c2a379696aaff34`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:35.979120 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9aa0e068-945f-4761-9e76-40ac7c96e5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:36.073270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1094/2210] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:25:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:36.078213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424739d0-2937-4ae4-a37a-89d534fca059 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:36.079773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-424739d0-2937-4ae4-a37a-89d534fca059 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:36.160258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424739d0-2937-4ae4-a37a-89d534fca059 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:36.165433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f514e-5e6e-42c3-bddd-c14c573f7304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:36.168944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f514e-5e6e-42c3-bddd-c14c573f7304 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:36.168944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f514e-5e6e-42c3-bddd-c14c573f7304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:36.168944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f514e-5e6e-42c3-bddd-c14c573f7304 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:36.168944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f514e-5e6e-42c3-bddd-c14c573f7304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:36.170156 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f514e-5e6e-42c3-bddd-c14c573f7304 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:36.277036 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1117/2211] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:25:36 2022] POST /identity/v3/projects => generated 377 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:25:36.282382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0e068-945f-4761-9e76-40ac7c96e5f5 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:36.294353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0e068-945f-4761-9e76-40ac7c96e5f5 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:36.359291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd155f3-4009-4aff-a9ec-e7bee8c2ceb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:36.359961 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd155f3-4009-4aff-a9ec-e7bee8c2ceb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:36.360247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd155f3-4009-4aff-a9ec-e7bee8c2ceb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:36.360525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd155f3-4009-4aff-a9ec-e7bee8c2ceb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:36.361323 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd155f3-4009-4aff-a9ec-e7bee8c2ceb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:36.363291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd155f3-4009-4aff-a9ec-e7bee8c2ceb5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:36.436522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1095/2212] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:25:36 2022] POST /identity/v3/users => generated 295 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:25:36.440714 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f514e-5e6e-42c3-bddd-c14c573f7304 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:36.451683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f514e-5e6e-42c3-bddd-c14c573f7304 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:36.456974 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54bb2-9981-4f3b-9574-8341a7a16c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:36.457645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54bb2-9981-4f3b-9574-8341a7a16c2d admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:36.457950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54bb2-9981-4f3b-9574-8341a7a16c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:36.458192 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54bb2-9981-4f3b-9574-8341a7a16c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:36.459039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf54bb2-9981-4f3b-9574-8341a7a16c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:36.460647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf54bb2-9981-4f3b-9574-8341a7a16c2d admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:36.473369 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1118/2213] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:25:36.478847 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd155f3-4009-4aff-a9ec-e7bee8c2ceb5 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:36.498261 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd155f3-4009-4aff-a9ec-e7bee8c2ceb5 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:36.507671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3864d-da08-4733-8edb-4b1f00af1768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:36.508172 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3864d-da08-4733-8edb-4b1f00af1768 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:36.508682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3864d-da08-4733-8edb-4b1f00af1768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:36.509733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3864d-da08-4733-8edb-4b1f00af1768 admin admin] PATH_INFO: `/v3/projects/dc3d8fedfb0c4d7c8742e5a1587ccadc/users/d35becd699f947afa0b5305e3cb7f0c2/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:36.538135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3864d-da08-4733-8edb-4b1f00af1768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc3d8fedfb0c4d7c8742e5a1587ccadc, user_id=d35becd699f947afa0b5305e3cb7f0c2, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:36.542025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3864d-da08-4733-8edb-4b1f00af1768 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:36.631750 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1096/2214] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:25:36 2022] PUT /identity/v3/projects/dc3d8fedfb0c4d7c8742e5a1587ccadc/users/d35becd699f947afa0b5305e3cb7f0c2/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:25:36.636210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54bb2-9981-4f3b-9574-8341a7a16c2d admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:36.649786 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54bb2-9981-4f3b-9574-8341a7a16c2d admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:36.666318 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:36.718053 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6054d-61e8-4a6e-ab13-b69c2af04487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:36.718279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6054d-61e8-4a6e-ab13-b69c2af04487 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:36.718508 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6054d-61e8-4a6e-ab13-b69c2af04487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:36.719316 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6054d-61e8-4a6e-ab13-b69c2af04487 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:36.719525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6054d-61e8-4a6e-ab13-b69c2af04487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:36.721625 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6054d-61e8-4a6e-ab13-b69c2af04487 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:36.742785 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1119/2215] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:25:36.747731 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3864d-da08-4733-8edb-4b1f00af1768 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:36.760471 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3864d-da08-4733-8edb-4b1f00af1768 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:36.766004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563436f-c6bc-440e-8770-ad3b3aed23d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:36.766990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563436f-c6bc-440e-8770-ad3b3aed23d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:36.767797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563436f-c6bc-440e-8770-ad3b3aed23d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:36.768180 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563436f-c6bc-440e-8770-ad3b3aed23d5 admin admin] PATH_INFO: `/v3/projects/dc3d8fedfb0c4d7c8742e5a1587ccadc/users/d35becd699f947afa0b5305e3cb7f0c2/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:36.770304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563436f-c6bc-440e-8770-ad3b3aed23d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc3d8fedfb0c4d7c8742e5a1587ccadc, user_id=d35becd699f947afa0b5305e3cb7f0c2, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:36.773123 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563436f-c6bc-440e-8770-ad3b3aed23d5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:36.784923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1097/2216] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:25:36 2022] PUT /identity/v3/projects/dc3d8fedfb0c4d7c8742e5a1587ccadc/users/d35becd699f947afa0b5305e3cb7f0c2/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:25:42.945806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667198af-9778-4840-b57c-6d55f031bca0 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:42.946336 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667198af-9778-4840-b57c-6d55f031bca0 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:42.946440 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667198af-9778-4840-b57c-6d55f031bca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:42.978147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-667198af-9778-4840-b57c-6d55f031bca0 None None] MFA Rules not processed for user `d35becd699f947afa0b5305e3cb7f0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:25:42.984867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-667198af-9778-4840-b57c-6d55f031bca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:43.060134 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1120/2217] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:25:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:25:43.074195 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563436f-c6bc-440e-8770-ad3b3aed23d5 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:43.076134 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a563436f-c6bc-440e-8770-ad3b3aed23d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:43.128742 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:43.160309 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563436f-c6bc-440e-8770-ad3b3aed23d5 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:43.166782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c71bb9a-6688-479c-82f7-bdbf9fa1a88b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:43.172396 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71bb9a-6688-479c-82f7-bdbf9fa1a88b service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:43.172396 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71bb9a-6688-479c-82f7-bdbf9fa1a88b service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:43.172396 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c71bb9a-6688-479c-82f7-bdbf9fa1a88b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:43.230492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71bb9a-6688-479c-82f7-bdbf9fa1a88b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:43.232192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c71bb9a-6688-479c-82f7-bdbf9fa1a88b service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:43.245941 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1098/2218] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:51.876376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667198af-9778-4840-b57c-6d55f031bca0 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:25:51.879233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-667198af-9778-4840-b57c-6d55f031bca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:51.945445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667198af-9778-4840-b57c-6d55f031bca0 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:25:51.949675 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5f094-1c15-4614-8253-5db40cfc73f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:25:51.950721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5f094-1c15-4614-8253-5db40cfc73f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:25:51.951796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5f094-1c15-4614-8253-5db40cfc73f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:25:51.952226 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5f094-1c15-4614-8253-5db40cfc73f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:25:51.954587 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b4e5f094-1c15-4614-8253-5db40cfc73f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:25:52.007898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:25:52.022544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5f094-1c15-4614-8253-5db40cfc73f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:25:52.024030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e5f094-1c15-4614-8253-5db40cfc73f2 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:25:52.034906 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1121/2219] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:25:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:25:53.627226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=37 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:26:02.057645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=6 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:26:16.015311 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2b590-6d63-4a55-954c-6f02443b1010 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:16.015880 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2b590-6d63-4a55-954c-6f02443b1010 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:16.015880 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2b590-6d63-4a55-954c-6f02443b1010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:16.037474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-caa2b590-6d63-4a55-954c-6f02443b1010 None None] MFA Rules not processed for user `4beff9397579496c80fd6e93b7a81542`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:26:16.038723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-caa2b590-6d63-4a55-954c-6f02443b1010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:16.104682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1099/2220] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:26:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:26:26.096298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=25 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:26:33.071708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1881fa-d7aa-4134-8729-b244ad73b93c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:33.072280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1881fa-d7aa-4134-8729-b244ad73b93c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:33.072280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1881fa-d7aa-4134-8729-b244ad73b93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:33.093052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-ee1881fa-d7aa-4134-8729-b244ad73b93c None None] MFA Rules not processed for user `0ac3d9b2893b4f078c2a379696aaff34`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:26:33.095112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ee1881fa-d7aa-4134-8729-b244ad73b93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:33.107647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1122/2221] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:26:43.086255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:26:47.282879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2b590-6d63-4a55-954c-6f02443b1010 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:26:47.293805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2b590-6d63-4a55-954c-6f02443b1010 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:26:47.297481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befe9ed4-dd4a-4cdb-9f82-0eb868d47251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:26:47.298141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe9ed4-dd4a-4cdb-9f82-0eb868d47251 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:47.298427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe9ed4-dd4a-4cdb-9f82-0eb868d47251 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:47.298704 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befe9ed4-dd4a-4cdb-9f82-0eb868d47251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:47.301259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-befe9ed4-dd4a-4cdb-9f82-0eb868d47251 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:47.315648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe9ed4-dd4a-4cdb-9f82-0eb868d47251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:26:47.317123 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befe9ed4-dd4a-4cdb-9f82-0eb868d47251 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:26:47.329409 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1100/2222] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:26:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:26:47.574705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1881fa-d7aa-4134-8729-b244ad73b93c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:26:47.577556 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ee1881fa-d7aa-4134-8729-b244ad73b93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:47.606408 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1881fa-d7aa-4134-8729-b244ad73b93c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:26:47.614873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c29b39-2520-4f9e-855d-198c3a8ea7dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:26:47.616675 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c29b39-2520-4f9e-855d-198c3a8ea7dd service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:47.617888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c29b39-2520-4f9e-855d-198c3a8ea7dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:47.618733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c29b39-2520-4f9e-855d-198c3a8ea7dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:47.622219 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-30c29b39-2520-4f9e-855d-198c3a8ea7dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:47.639003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c29b39-2520-4f9e-855d-198c3a8ea7dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:26:47.640694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c29b39-2520-4f9e-855d-198c3a8ea7dd service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:26:47.657473 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1123/2223] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:26:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:26:50.419841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a647b-69e0-45dc-a90b-ad6028ab86a8 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:50.420359 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a647b-69e0-45dc-a90b-ad6028ab86a8 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:50.420523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a647b-69e0-45dc-a90b-ad6028ab86a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:50.439240 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-af3a647b-69e0-45dc-a90b-ad6028ab86a8 None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:26:50.440762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-af3a647b-69e0-45dc-a90b-ad6028ab86a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:50.505088 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1101/2224] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jun 15 10:26:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:26:50.692160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c29b39-2520-4f9e-855d-198c3a8ea7dd service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:26:50.694110 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-30c29b39-2520-4f9e-855d-198c3a8ea7dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:50.781270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c29b39-2520-4f9e-855d-198c3a8ea7dd service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:26:50.788732 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5006dd6f-b036-4fd1-bd16-45776e21f9c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:26:50.789442 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006dd6f-b036-4fd1-bd16-45776e21f9c5 service swift] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:50.789728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006dd6f-b036-4fd1-bd16-45776e21f9c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:50.790004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5006dd6f-b036-4fd1-bd16-45776e21f9c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:50.800868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006dd6f-b036-4fd1-bd16-45776e21f9c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:26:50.802851 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5006dd6f-b036-4fd1-bd16-45776e21f9c5 service swift] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:26:50.812745 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1124/2225] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 10:26:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:26:56.920482 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a647b-69e0-45dc-a90b-ad6028ab86a8 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:26:57.039212 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a647b-69e0-45dc-a90b-ad6028ab86a8 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:26:57.044657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbdb98-e441-4fae-a24d-3092d49a32a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:26:57.045723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbdb98-e441-4fae-a24d-3092d49a32a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:57.046253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbdb98-e441-4fae-a24d-3092d49a32a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:57.047236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbdb98-e441-4fae-a24d-3092d49a32a1 admin admin] PATH_INFO: `/v3/users/3aac0bb570d24c069b7467be64085e2f` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:57.049110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbdb98-e441-4fae-a24d-3092d49a32a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aac0bb570d24c069b7467be64085e2f)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:26:57.052418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbdb98-e441-4fae-a24d-3092d49a32a1 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:26:57.266394 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-2adbdb98-e441-4fae-a24d-3092d49a32a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aac0bb570d24c069b7467be64085e2f'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:26:57.338763 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-2adbdb98-e441-4fae-a24d-3092d49a32a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aac0bb570d24c069b7467be64085e2f'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:26:57.339815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-2adbdb98-e441-4fae-a24d-3092d49a32a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aac0bb570d24c069b7467be64085e2f'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:26:57.340677 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-2adbdb98-e441-4fae-a24d-3092d49a32a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aac0bb570d24c069b7467be64085e2f'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:26:57.358039 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-2adbdb98-e441-4fae-a24d-3092d49a32a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aac0bb570d24c069b7467be64085e2f'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:26:57.378263 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1102/2226] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:26:56 2022] DELETE /identity/v3/users/3aac0bb570d24c069b7467be64085e2f => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:26:57.380642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239be61d-497b-4f74-94c1-a7f7d5f67c75 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:57.381214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239be61d-497b-4f74-94c1-a7f7d5f67c75 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:57.381673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239be61d-497b-4f74-94c1-a7f7d5f67c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:57.411627 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-239be61d-497b-4f74-94c1-a7f7d5f67c75 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:26:57.413743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-239be61d-497b-4f74-94c1-a7f7d5f67c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:57.583447 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1125/2227] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:26:57.592010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbdb98-e441-4fae-a24d-3092d49a32a1 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:26:57.593827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-2adbdb98-e441-4fae-a24d-3092d49a32a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:57.618523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:26:57.742710 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbdb98-e441-4fae-a24d-3092d49a32a1 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:26:57.747125 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431d417-c8d3-4302-b59b-f620098c8b21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:26:57.748253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431d417-c8d3-4302-b59b-f620098c8b21 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:57.748658 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431d417-c8d3-4302-b59b-f620098c8b21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:57.749074 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431d417-c8d3-4302-b59b-f620098c8b21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:57.765193 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431d417-c8d3-4302-b59b-f620098c8b21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:26:57.766820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431d417-c8d3-4302-b59b-f620098c8b21 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:26:57.793277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1103/2228] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:26:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:26:58.981659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239be61d-497b-4f74-94c1-a7f7d5f67c75 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:26:58.983581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-239be61d-497b-4f74-94c1-a7f7d5f67c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:59.002089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239be61d-497b-4f74-94c1-a7f7d5f67c75 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:26:59.010674 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48451ac-f195-4a53-be33-acb3f29fd4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:26:59.018577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48451ac-f195-4a53-be33-acb3f29fd4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:59.018577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48451ac-f195-4a53-be33-acb3f29fd4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:59.018577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48451ac-f195-4a53-be33-acb3f29fd4db admin admin] PATH_INFO: `/v3/projects/85bf4851de0647b4a51b5b3699f4b206` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:59.018577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48451ac-f195-4a53-be33-acb3f29fd4db admin admin] RBAC: Authorizing `identity:delete_project(project_id=85bf4851de0647b4a51b5b3699f4b206)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:26:59.023389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48451ac-f195-4a53-be33-acb3f29fd4db admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:26:59.272775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e48451ac-f195-4a53-be33-acb3f29fd4db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85bf4851de0647b4a51b5b3699f4b206', 'request_id': 'req-e48451ac-f195-4a53-be33-acb3f29fd4db'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:26:59.294728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e48451ac-f195-4a53-be33-acb3f29fd4db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85bf4851de0647b4a51b5b3699f4b206', 'request_id': 'req-e48451ac-f195-4a53-be33-acb3f29fd4db'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:26:59.314207 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e48451ac-f195-4a53-be33-acb3f29fd4db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85bf4851de0647b4a51b5b3699f4b206', 'request_id': 'req-e48451ac-f195-4a53-be33-acb3f29fd4db'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:26:59.426489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:26:59.440543 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e48451ac-f195-4a53-be33-acb3f29fd4db admin admin] The token cache is being invalidate because project 85bf4851de0647b4a51b5b3699f4b206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:26:59.442373 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e48451ac-f195-4a53-be33-acb3f29fd4db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:26:59.454926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e48451ac-f195-4a53-be33-acb3f29fd4db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:26:59.467551 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1126/2229] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:26:58 2022] DELETE /identity/v3/projects/85bf4851de0647b4a51b5b3699f4b206 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:26:59.475524 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07771d89-a446-4368-a139-9c02e29f7c48 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:59.476242 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07771d89-a446-4368-a139-9c02e29f7c48 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:59.476242 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07771d89-a446-4368-a139-9c02e29f7c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:59.556247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-07771d89-a446-4368-a139-9c02e29f7c48 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:26:59.558419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-07771d89-a446-4368-a139-9c02e29f7c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:59.715258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1104/2230] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:26:59.729761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24358a-0dd9-4afa-907b-36aad80bbd68 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:59.730243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24358a-0dd9-4afa-907b-36aad80bbd68 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:59.730709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24358a-0dd9-4afa-907b-36aad80bbd68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:59.775393 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-0c24358a-0dd9-4afa-907b-36aad80bbd68 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:26:59.777557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0c24358a-0dd9-4afa-907b-36aad80bbd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:26:59.795616 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1127/2231] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:26:59.802222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07771d89-a446-4368-a139-9c02e29f7c48 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:26:59.830273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07771d89-a446-4368-a139-9c02e29f7c48 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:26:59.842759 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b271a6-b191-4c83-a9be-7bdee2eb34c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:26:59.843821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b271a6-b191-4c83-a9be-7bdee2eb34c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:59.844409 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b271a6-b191-4c83-a9be-7bdee2eb34c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:59.844987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b271a6-b191-4c83-a9be-7bdee2eb34c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:59.846601 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b271a6-b191-4c83-a9be-7bdee2eb34c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 10:26:59.847167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b271a6-b191-4c83-a9be-7bdee2eb34c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:26:59.850568 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b271a6-b191-4c83-a9be-7bdee2eb34c3 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:26:59.884329 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1105/2232] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 10:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 10:26:59.891645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24358a-0dd9-4afa-907b-36aad80bbd68 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:26:59.920270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24358a-0dd9-4afa-907b-36aad80bbd68 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:26:59.929235 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e5eb-053b-4fea-b046-d2612de01693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:26:59.930616 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e5eb-053b-4fea-b046-d2612de01693 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:26:59.931294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e5eb-053b-4fea-b046-d2612de01693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:26:59.931948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e5eb-053b-4fea-b046-d2612de01693 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:26:59.933334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e5eb-053b-4fea-b046-d2612de01693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:26:59.937441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e5eb-053b-4fea-b046-d2612de01693 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:00.071453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1128/2233] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 10:26:59 2022] POST /identity/v3/projects => generated 373 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 10:27:00.078179 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b271a6-b191-4c83-a9be-7bdee2eb34c3 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:00.108340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b271a6-b191-4c83-a9be-7bdee2eb34c3 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:00.242402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d66c7a-3f01-418f-b191-1c0750878399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:00.243142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d66c7a-3f01-418f-b191-1c0750878399 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:00.243431 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d66c7a-3f01-418f-b191-1c0750878399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:00.243723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d66c7a-3f01-418f-b191-1c0750878399 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:00.244398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d66c7a-3f01-418f-b191-1c0750878399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:00.247060 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d66c7a-3f01-418f-b191-1c0750878399 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:00.406098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1106/2234] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 10:27:00 2022] POST /identity/v3/users => generated 287 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 10:27:00.415331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e5eb-053b-4fea-b046-d2612de01693 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:00.456445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e5eb-053b-4fea-b046-d2612de01693 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:00.482645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261803c-32fb-4aba-abaf-d28902103b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:00.483738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261803c-32fb-4aba-abaf-d28902103b38 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:00.484233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261803c-32fb-4aba-abaf-d28902103b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:00.484725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261803c-32fb-4aba-abaf-d28902103b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:00.488783 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261803c-32fb-4aba-abaf-d28902103b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:00.493611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261803c-32fb-4aba-abaf-d28902103b38 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:00.527210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1129/2235] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 10:27:00.531911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d66c7a-3f01-418f-b191-1c0750878399 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:00.566318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d66c7a-3f01-418f-b191-1c0750878399 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:00.588752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4080a8d-0d96-4c5f-9862-0af8c612e94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:00.591544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4080a8d-0d96-4c5f-9862-0af8c612e94b admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:00.592131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4080a8d-0d96-4c5f-9862-0af8c612e94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:00.592686 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4080a8d-0d96-4c5f-9862-0af8c612e94b admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:00.594509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4080a8d-0d96-4c5f-9862-0af8c612e94b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:00.597786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4080a8d-0d96-4c5f-9862-0af8c612e94b admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:00.623129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1107/2236] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 10:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 10:27:00.625140 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261803c-32fb-4aba-abaf-d28902103b38 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:00.650465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261803c-32fb-4aba-abaf-d28902103b38 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:00.661521 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705301f-d699-4256-b579-6a4d74bf65df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:00.663224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705301f-d699-4256-b579-6a4d74bf65df admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:00.663724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705301f-d699-4256-b579-6a4d74bf65df admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:00.664227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705301f-d699-4256-b579-6a4d74bf65df admin admin] PATH_INFO: `/v3/projects/57b2cd6b94be49228fec6c9f92d1ccc1/users/c0449a8072f9479d827fb655bb0383e3/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:00.697292 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705301f-d699-4256-b579-6a4d74bf65df admin admin] RBAC: Authorizing `identity:create_grant(project_id=57b2cd6b94be49228fec6c9f92d1ccc1, user_id=c0449a8072f9479d827fb655bb0383e3, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:00.703617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705301f-d699-4256-b579-6a4d74bf65df admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:00.791274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1130/2237] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 10:27:00 2022] PUT /identity/v3/projects/57b2cd6b94be49228fec6c9f92d1ccc1/users/c0449a8072f9479d827fb655bb0383e3/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:27:00.793280 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d858e-20b9-419b-9a3a-e1d3fb50ba14 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:00.793822 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d858e-20b9-419b-9a3a-e1d3fb50ba14 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:00.794276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1d858e-20b9-419b-9a3a-e1d3fb50ba14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:00.886835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-3e1d858e-20b9-419b-9a3a-e1d3fb50ba14 None None] MFA Rules not processed for user `c0449a8072f9479d827fb655bb0383e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:27:00.889024 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3e1d858e-20b9-419b-9a3a-e1d3fb50ba14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:00.905529 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:27:01.031096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1108/2238] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:27:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:27:01.032560 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705301f-d699-4256-b579-6a4d74bf65df admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:01.036710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5705301f-d699-4256-b579-6a4d74bf65df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:01.177567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705301f-d699-4256-b579-6a4d74bf65df admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:01.181260 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d6ee8-817e-44b3-a53e-c7dd8b10b53c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:01.182424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d6ee8-817e-44b3-a53e-c7dd8b10b53c service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:01.183104 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d6ee8-817e-44b3-a53e-c7dd8b10b53c service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:01.183426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d6ee8-817e-44b3-a53e-c7dd8b10b53c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:01.210265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d6ee8-817e-44b3-a53e-c7dd8b10b53c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:01.213320 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d6ee8-817e-44b3-a53e-c7dd8b10b53c service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:01.232509 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1131/2239] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:27:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:27:04.425621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da5839-7117-4f5c-8102-3fccc7d1f8b1 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:04.426183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da5839-7117-4f5c-8102-3fccc7d1f8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:04.426183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da5839-7117-4f5c-8102-3fccc7d1f8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:04.447007 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-67da5839-7117-4f5c-8102-3fccc7d1f8b1 None None] MFA Rules not processed for user `c0449a8072f9479d827fb655bb0383e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:27:04.448402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-67da5839-7117-4f5c-8102-3fccc7d1f8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:04.464632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1109/2240] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:27:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:27:05.115793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d6ee8-817e-44b3-a53e-c7dd8b10b53c service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:05.117678 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-aa0d6ee8-817e-44b3-a53e-c7dd8b10b53c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:05.210323 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d6ee8-817e-44b3-a53e-c7dd8b10b53c service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:05.219800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8a02b-d336-4360-b5e8-8f8c9453d482 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:05.220666 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8a02b-d336-4360-b5e8-8f8c9453d482 service placement] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:05.222111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8a02b-d336-4360-b5e8-8f8c9453d482 service placement] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:05.222631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8a02b-d336-4360-b5e8-8f8c9453d482 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:05.226042 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9de8a02b-d336-4360-b5e8-8f8c9453d482 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:05.247211 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8a02b-d336-4360-b5e8-8f8c9453d482 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:05.250183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de8a02b-d336-4360-b5e8-8f8c9453d482 service placement] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:05.270655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1132/2241] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:27:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:27:05.631236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da5839-7117-4f5c-8102-3fccc7d1f8b1 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:05.632722 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-67da5839-7117-4f5c-8102-3fccc7d1f8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:05.714745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da5839-7117-4f5c-8102-3fccc7d1f8b1 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:05.720753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46485de-50c6-49da-b89f-d2e7e84aeb22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:05.721964 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46485de-50c6-49da-b89f-d2e7e84aeb22 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:05.722242 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46485de-50c6-49da-b89f-d2e7e84aeb22 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:05.722543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46485de-50c6-49da-b89f-d2e7e84aeb22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:05.724433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c46485de-50c6-49da-b89f-d2e7e84aeb22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:05.797938 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46485de-50c6-49da-b89f-d2e7e84aeb22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:05.800531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46485de-50c6-49da-b89f-d2e7e84aeb22 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:05.809859 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1110/2242] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:27:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:27:10.635433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8a02b-d336-4360-b5e8-8f8c9453d482 service placement] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:10.637917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9de8a02b-d336-4360-b5e8-8f8c9453d482 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:10.686704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:27:10.734236 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de8a02b-d336-4360-b5e8-8f8c9453d482 service placement] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:10.739011 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c275e3-cdb5-4fef-bb00-40e52ccc2748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:10.739969 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c275e3-cdb5-4fef-bb00-40e52ccc2748 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:10.740408 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c275e3-cdb5-4fef-bb00-40e52ccc2748 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:10.741076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c275e3-cdb5-4fef-bb00-40e52ccc2748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:10.743278 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f8c275e3-cdb5-4fef-bb00-40e52ccc2748 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:10.758902 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c275e3-cdb5-4fef-bb00-40e52ccc2748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:10.760710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c275e3-cdb5-4fef-bb00-40e52ccc2748 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:10.772080 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1133/2243] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:27:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:27:15.805127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=82 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:27:16.794738 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b26a89-9f1e-4c27-97a2-1a3df779311b None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:16.797362 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b26a89-9f1e-4c27-97a2-1a3df779311b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:16.798112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b26a89-9f1e-4c27-97a2-1a3df779311b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:16.835636 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-78b26a89-9f1e-4c27-97a2-1a3df779311b None None] MFA Rules not processed for user `80a318aa610e449fb6e480372ee61db8`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:27:16.838728 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-78b26a89-9f1e-4c27-97a2-1a3df779311b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:17.024111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1111/2244] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:27:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:27:17.499165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c275e3-cdb5-4fef-bb00-40e52ccc2748 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:17.516690 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c275e3-cdb5-4fef-bb00-40e52ccc2748 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:17.524124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02866fe-5e6f-4a92-9641-69404c8a71f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:17.525105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02866fe-5e6f-4a92-9641-69404c8a71f7 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:17.525429 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02866fe-5e6f-4a92-9641-69404c8a71f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:17.525654 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02866fe-5e6f-4a92-9641-69404c8a71f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:17.528187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f02866fe-5e6f-4a92-9641-69404c8a71f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:17.543389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02866fe-5e6f-4a92-9641-69404c8a71f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:17.549997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02866fe-5e6f-4a92-9641-69404c8a71f7 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:17.572436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1134/2245] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:27:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:27:17.788652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b26a89-9f1e-4c27-97a2-1a3df779311b None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:17.799237 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b26a89-9f1e-4c27-97a2-1a3df779311b None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:17.803397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c733e88-e245-4d9d-9c2f-9d26df0929fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:17.804065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c733e88-e245-4d9d-9c2f-9d26df0929fd service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:17.804345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c733e88-e245-4d9d-9c2f-9d26df0929fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:17.804617 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c733e88-e245-4d9d-9c2f-9d26df0929fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:17.806549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9c733e88-e245-4d9d-9c2f-9d26df0929fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:17.818221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c733e88-e245-4d9d-9c2f-9d26df0929fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:17.819874 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c733e88-e245-4d9d-9c2f-9d26df0929fd service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:17.829977 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1112/2246] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:27:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:27:20.337041 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02866fe-5e6f-4a92-9641-69404c8a71f7 service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:20.338699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f02866fe-5e6f-4a92-9641-69404c8a71f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:20.354393 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02866fe-5e6f-4a92-9641-69404c8a71f7 service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:20.358665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6f5cd-3240-47a0-94e4-66d63c429109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:20.360289 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6f5cd-3240-47a0-94e4-66d63c429109 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:20.360289 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6f5cd-3240-47a0-94e4-66d63c429109 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:20.360289 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6f5cd-3240-47a0-94e4-66d63c429109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:20.362588 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e0e6f5cd-3240-47a0-94e4-66d63c429109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:20.379450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6f5cd-3240-47a0-94e4-66d63c429109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:20.380947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6f5cd-3240-47a0-94e4-66d63c429109 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:20.391360 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1135/2247] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:27:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:27:20.834363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28729449-715d-4400-8626-cca0d8b090ac None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:20.835197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28729449-715d-4400-8626-cca0d8b090ac None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:20.835197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28729449-715d-4400-8626-cca0d8b090ac None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:20.836475 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1113/2248] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jun 15 10:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 15 10:27:20.850256 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a5d69-7759-46e4-8094-3d74a29dd2a9 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:20.850728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a5d69-7759-46e4-8094-3d74a29dd2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:20.851240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a5d69-7759-46e4-8094-3d74a29dd2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:20.906184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-d10a5d69-7759-46e4-8094-3d74a29dd2a9 None None] MFA Rules not processed for user `26f23f8595fc43ad94aee7d0192164a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:27:20.907470 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d10a5d69-7759-46e4-8094-3d74a29dd2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:20.920874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1136/2249] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jun 15 10:27:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:27:20.938088 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28729449-715d-4400-8626-cca0d8b090ac None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:20.942105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-28729449-715d-4400-8626-cca0d8b090ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:21.095469 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28729449-715d-4400-8626-cca0d8b090ac None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:21.103490 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc58180-5bbc-4e3d-92f1-9d27fa14bfa8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fbf33189aa94f4f973d558d615a23e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:21.104711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc58180-5bbc-4e3d-92f1-9d27fa14bfa8 service glance] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:21.105311 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc58180-5bbc-4e3d-92f1-9d27fa14bfa8 service glance] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:21.105968 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc58180-5bbc-4e3d-92f1-9d27fa14bfa8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:21.129421 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc58180-5bbc-4e3d-92f1-9d27fa14bfa8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:21.132470 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc58180-5bbc-4e3d-92f1-9d27fa14bfa8 service glance] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:21.158054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1114/2250] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:27:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:27:21.353943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a5d69-7759-46e4-8094-3d74a29dd2a9 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:21.364775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a5d69-7759-46e4-8094-3d74a29dd2a9 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:21.368949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a70cb-8d9e-4694-9c25-e53207c7ef3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:21.371417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a70cb-8d9e-4694-9c25-e53207c7ef3a service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:21.371417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a70cb-8d9e-4694-9c25-e53207c7ef3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:21.371417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a70cb-8d9e-4694-9c25-e53207c7ef3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:21.372984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-8a4a70cb-8d9e-4694-9c25-e53207c7ef3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:21.438468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee32132-88b1-481d-aa7c-07cf9e3b0411 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:21.439043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee32132-88b1-481d-aa7c-07cf9e3b0411 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:21.439530 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee32132-88b1-481d-aa7c-07cf9e3b0411 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:21.442743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1115/2251] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jun 15 10:27:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:27:21.450500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b84b60-92bc-4b63-9488-6ee618f67381 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:21.451085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b84b60-92bc-4b63-9488-6ee618f67381 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:21.451606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b84b60-92bc-4b63-9488-6ee618f67381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:21.463991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a70cb-8d9e-4694-9c25-e53207c7ef3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:21.465517 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a70cb-8d9e-4694-9c25-e53207c7ef3a service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:21.477859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1137/2252] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:27:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:27:21.482755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a1b84b60-92bc-4b63-9488-6ee618f67381 None None] MFA Rules not processed for user `ae926ce83d464c768baddd90de31aa40`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:27:21.483995 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a1b84b60-92bc-4b63-9488-6ee618f67381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:21.493667 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1116/2253] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jun 15 10:27:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:27:21.509223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a70cb-8d9e-4694-9c25-e53207c7ef3a service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:21.522608 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a70cb-8d9e-4694-9c25-e53207c7ef3a service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:21.527499 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e671c-923c-4911-8b3a-ddbf952eb1e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:21.528271 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e671c-923c-4911-8b3a-ddbf952eb1e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:21.528626 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e671c-923c-4911-8b3a-ddbf952eb1e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:21.529915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e671c-923c-4911-8b3a-ddbf952eb1e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:21.543170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e671c-923c-4911-8b3a-ddbf952eb1e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:21.546176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e671c-923c-4911-8b3a-ddbf952eb1e9 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:21.562150 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1138/2254] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:27:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:27:21.662137 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01912b89-f6c4-4078-8fb1-f59a09a40e9f None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:21.662825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01912b89-f6c4-4078-8fb1-f59a09a40e9f None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:21.663358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01912b89-f6c4-4078-8fb1-f59a09a40e9f None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:21.664993 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1117/2255] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jun 15 10:27:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:27:21.671464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bc547-f586-4c1a-b315-cb117a336ee1 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:21.676659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bc547-f586-4c1a-b315-cb117a336ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:21.676659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bc547-f586-4c1a-b315-cb117a336ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:21.701270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-3a2bc547-f586-4c1a-b315-cb117a336ee1 None None] MFA Rules not processed for user `26f23f8595fc43ad94aee7d0192164a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:27:21.703485 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-3a2bc547-f586-4c1a-b315-cb117a336ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:21.715862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1139/2256] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jun 15 10:27:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:27:21.728254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01912b89-f6c4-4078-8fb1-f59a09a40e9f None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:21.749910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01912b89-f6c4-4078-8fb1-f59a09a40e9f None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:21.758114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0b077-dd33-476b-9400-0f0d04362db8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:21.759171 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0b077-dd33-476b-9400-0f0d04362db8 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:21.759638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0b077-dd33-476b-9400-0f0d04362db8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:21.760138 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0b077-dd33-476b-9400-0f0d04362db8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:21.776185 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0b077-dd33-476b-9400-0f0d04362db8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:21.776185 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0b077-dd33-476b-9400-0f0d04362db8 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:21.796220 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1118/2257] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:27:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:27:30.999898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbf487-010e-4028-8986-43f0431fcb36 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:31.000731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbf487-010e-4028-8986-43f0431fcb36 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:31.000731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbf487-010e-4028-8986-43f0431fcb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:31.029310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-62cbf487-010e-4028-8986-43f0431fcb36 None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:27:31.030549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-62cbf487-010e-4028-8986-43f0431fcb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:31.095298 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1140/2258] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jun 15 10:27:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:27:31.108819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0b077-dd33-476b-9400-0f0d04362db8 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:27:31.112909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-7bc0b077-dd33-476b-9400-0f0d04362db8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:27:31.203596 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:27:31.221404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0b077-dd33-476b-9400-0f0d04362db8 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:27:31.229386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5ad58-e194-4ad9-9657-313ee2688ed8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:27:31.231617 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5ad58-e194-4ad9-9657-313ee2688ed8 service swift] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:27:31.231617 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5ad58-e194-4ad9-9657-313ee2688ed8 service swift] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:27:31.231617 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5ad58-e194-4ad9-9657-313ee2688ed8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:27:31.244615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5ad58-e194-4ad9-9657-313ee2688ed8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:27:31.247969 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5ad58-e194-4ad9-9657-313ee2688ed8 service swift] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:27:31.262462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1119/2259] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 10:27:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:27:41.088124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=98 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:27:41.259137 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=7 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:28:02.879118 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbf487-010e-4028-8986-43f0431fcb36 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:28:02.901040 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbf487-010e-4028-8986-43f0431fcb36 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:28:02.907872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f45d8-8e31-44be-a136-2f3f52f8912a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:28:02.909652 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f45d8-8e31-44be-a136-2f3f52f8912a service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:02.910090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f45d8-8e31-44be-a136-2f3f52f8912a service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:02.910383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f45d8-8e31-44be-a136-2f3f52f8912a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:02.913430 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7c7f45d8-8e31-44be-a136-2f3f52f8912a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:28:02.928010 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f45d8-8e31-44be-a136-2f3f52f8912a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:28:02.929716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f45d8-8e31-44be-a136-2f3f52f8912a service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:28:02.942900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1141/2260] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:28:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:28:08.628400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1baee4-1d9e-4a91-8c44-b731233eeb7e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:08.628400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1baee4-1d9e-4a91-8c44-b731233eeb7e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:08.629185 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1baee4-1d9e-4a91-8c44-b731233eeb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:08.648000 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-6b1baee4-1d9e-4a91-8c44-b731233eeb7e None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:28:08.651684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-6b1baee4-1d9e-4a91-8c44-b731233eeb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:28:08.659273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1120/2261] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jun 15 10:28:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:28:08.687885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f45d8-8e31-44be-a136-2f3f52f8912a service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:28:08.704352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f45d8-8e31-44be-a136-2f3f52f8912a service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:28:08.715293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a541c9-eedc-4be8-bbe9-019dd203c654 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:28:08.715840 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a541c9-eedc-4be8-bbe9-019dd203c654 service swift] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:08.716125 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a541c9-eedc-4be8-bbe9-019dd203c654 service swift] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:08.716405 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a541c9-eedc-4be8-bbe9-019dd203c654 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:08.729659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a541c9-eedc-4be8-bbe9-019dd203c654 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:28:08.732881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a541c9-eedc-4be8-bbe9-019dd203c654 service swift] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:28:08.751144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1142/2262] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 10:28:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:28:09.785994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef6735-76f5-4a32-873a-67e14a0a0fae None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:09.786853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef6735-76f5-4a32-873a-67e14a0a0fae None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:09.786853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef6735-76f5-4a32-873a-67e14a0a0fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:09.814685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-1aef6735-76f5-4a32-873a-67e14a0a0fae None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:28:09.814685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1aef6735-76f5-4a32-873a-67e14a0a0fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:28:09.821430 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1121/2263] 10.222.0.60 () {58 vars in 967 bytes} [Wed Jun 15 10:28:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:28:09.838172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a541c9-eedc-4be8-bbe9-019dd203c654 service swift] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:28:09.855374 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a541c9-eedc-4be8-bbe9-019dd203c654 service swift] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:28:09.864090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bbe8f-47bd-4463-977a-a054fd6b8d90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:28:09.865488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bbe8f-47bd-4463-977a-a054fd6b8d90 service swift] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:09.866064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bbe8f-47bd-4463-977a-a054fd6b8d90 service swift] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:09.866602 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bbe8f-47bd-4463-977a-a054fd6b8d90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:09.883058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bbe8f-47bd-4463-977a-a054fd6b8d90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:28:09.886265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bbe8f-47bd-4463-977a-a054fd6b8d90 service swift] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:28:09.901176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1143/2264] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 10:28:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:28:10.554760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2c12a-94e2-42e5-a335-47f44a658515 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:10.555052 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2c12a-94e2-42e5-a335-47f44a658515 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:10.555329 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2c12a-94e2-42e5-a335-47f44a658515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:10.576262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-e0e2c12a-94e2-42e5-a335-47f44a658515 None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:28:10.577884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e0e2c12a-94e2-42e5-a335-47f44a658515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:28:10.587248 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1122/2265] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jun 15 10:28:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:28:10.600261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bbe8f-47bd-4463-977a-a054fd6b8d90 service swift] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:28:10.611767 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bbe8f-47bd-4463-977a-a054fd6b8d90 service swift] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:28:10.617280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1a27c-1320-4467-b735-730c161256a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:28:10.617901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1a27c-1320-4467-b735-730c161256a8 service swift] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:10.618275 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1a27c-1320-4467-b735-730c161256a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:10.618559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1a27c-1320-4467-b735-730c161256a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:10.628802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1a27c-1320-4467-b735-730c161256a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:28:10.630513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1a27c-1320-4467-b735-730c161256a8 service swift] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:28:10.640317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1144/2266] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 10:28:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:28:11.363999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72bf12-6499-4914-9b0b-c75c66fce6a4 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:11.364575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72bf12-6499-4914-9b0b-c75c66fce6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:11.365039 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72bf12-6499-4914-9b0b-c75c66fce6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:11.396752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-0e72bf12-6499-4914-9b0b-c75c66fce6a4 None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:28:11.399182 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-0e72bf12-6499-4914-9b0b-c75c66fce6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:28:11.412033 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1123/2267] 10.222.0.60 () {58 vars in 967 bytes} [Wed Jun 15 10:28:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:28:11.427336 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1a27c-1320-4467-b735-730c161256a8 service swift] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:28:11.440676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1a27c-1320-4467-b735-730c161256a8 service swift] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:28:11.446241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252c72b-5ea3-4389-82eb-8460f33017fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:28:11.446903 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252c72b-5ea3-4389-82eb-8460f33017fb service swift] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:11.447195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252c72b-5ea3-4389-82eb-8460f33017fb service swift] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:11.447482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252c72b-5ea3-4389-82eb-8460f33017fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:11.458155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252c72b-5ea3-4389-82eb-8460f33017fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:28:11.460098 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252c72b-5ea3-4389-82eb-8460f33017fb service swift] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:28:11.473572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1145/2268] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 10:28:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:28:13.623672 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd208d-afdd-4195-aae8-12540ec97e36 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:13.624488 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd208d-afdd-4195-aae8-12540ec97e36 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:13.629948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd208d-afdd-4195-aae8-12540ec97e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:13.651263 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-89bd208d-afdd-4195-aae8-12540ec97e36 None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:28:13.652864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-89bd208d-afdd-4195-aae8-12540ec97e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:28:13.662756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1124/2269] 10.222.0.60 () {58 vars in 967 bytes} [Wed Jun 15 10:28:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:28:13.679849 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252c72b-5ea3-4389-82eb-8460f33017fb service swift] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:28:13.699070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252c72b-5ea3-4389-82eb-8460f33017fb service swift] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:28:13.708920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e03f7-8acb-4a80-8dee-954d3cb4af03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:28:13.709964 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e03f7-8acb-4a80-8dee-954d3cb4af03 service swift] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:13.710352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e03f7-8acb-4a80-8dee-954d3cb4af03 service swift] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:13.710746 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e03f7-8acb-4a80-8dee-954d3cb4af03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:13.723852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e03f7-8acb-4a80-8dee-954d3cb4af03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:28:13.726138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e03f7-8acb-4a80-8dee-954d3cb4af03 service swift] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:28:13.736939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1146/2270] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 10:28:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 10:28:14.488226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd208d-afdd-4195-aae8-12540ec97e36 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:28:14.584637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd208d-afdd-4195-aae8-12540ec97e36 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:28:14.587486 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d5970-685b-45d9-87a3-2ec6a0621adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:28:14.588429 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d5970-685b-45d9-87a3-2ec6a0621adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:14.588714 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d5970-685b-45d9-87a3-2ec6a0621adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:14.589070 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d5970-685b-45d9-87a3-2ec6a0621adb admin admin] PATH_INFO: `/v3/users/c0449a8072f9479d827fb655bb0383e3` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:14.590681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d5970-685b-45d9-87a3-2ec6a0621adb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0449a8072f9479d827fb655bb0383e3)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:28:14.592639 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d5970-685b-45d9-87a3-2ec6a0621adb admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:28:14.827178 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-3c7d5970-685b-45d9-87a3-2ec6a0621adb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0449a8072f9479d827fb655bb0383e3'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:28:14.894273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-3c7d5970-685b-45d9-87a3-2ec6a0621adb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0449a8072f9479d827fb655bb0383e3'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:28:14.895095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-3c7d5970-685b-45d9-87a3-2ec6a0621adb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0449a8072f9479d827fb655bb0383e3'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:28:14.895773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-3c7d5970-685b-45d9-87a3-2ec6a0621adb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0449a8072f9479d827fb655bb0383e3'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:28:14.920223 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-3c7d5970-685b-45d9-87a3-2ec6a0621adb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0449a8072f9479d827fb655bb0383e3'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:28:14.941488 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1125/2271] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:28:14 2022] DELETE /identity/v3/users/c0449a8072f9479d827fb655bb0383e3 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:28:14.948101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f552186-5284-402a-ac58-cc52d1efc295 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:14.949175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f552186-5284-402a-ac58-cc52d1efc295 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:14.949619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f552186-5284-402a-ac58-cc52d1efc295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:14.972319 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-2f552186-5284-402a-ac58-cc52d1efc295 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:28:14.973865 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2f552186-5284-402a-ac58-cc52d1efc295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:28:15.046825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1147/2272] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:28:15.057730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d5970-685b-45d9-87a3-2ec6a0621adb admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:28:15.059244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3c7d5970-685b-45d9-87a3-2ec6a0621adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:28:15.124996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:28:15.131779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d5970-685b-45d9-87a3-2ec6a0621adb admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:28:15.137264 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bb874-50a2-4b9d-9934-6e88a6b56161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:28:15.137264 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bb874-50a2-4b9d-9934-6e88a6b56161 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:15.137264 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bb874-50a2-4b9d-9934-6e88a6b56161 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:15.137762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bb874-50a2-4b9d-9934-6e88a6b56161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:15.149060 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47bb874-50a2-4b9d-9934-6e88a6b56161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:28:15.150957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47bb874-50a2-4b9d-9934-6e88a6b56161 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:28:15.162859 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1126/2273] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:28:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:28:15.479839 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f552186-5284-402a-ac58-cc52d1efc295 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:28:15.481972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2f552186-5284-402a-ac58-cc52d1efc295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:28:15.496500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f552186-5284-402a-ac58-cc52d1efc295 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:28:15.504071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d131a6a-df90-4da8-b07a-5fcc5ef89256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:28:15.504999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d131a6a-df90-4da8-b07a-5fcc5ef89256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:15.505301 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d131a6a-df90-4da8-b07a-5fcc5ef89256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:15.505582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d131a6a-df90-4da8-b07a-5fcc5ef89256 admin admin] PATH_INFO: `/v3/projects/57b2cd6b94be49228fec6c9f92d1ccc1` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:15.506970 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d131a6a-df90-4da8-b07a-5fcc5ef89256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57b2cd6b94be49228fec6c9f92d1ccc1)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:28:15.509078 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d131a6a-df90-4da8-b07a-5fcc5ef89256 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:28:15.655691 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6d131a6a-df90-4da8-b07a-5fcc5ef89256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57b2cd6b94be49228fec6c9f92d1ccc1', 'request_id': 'req-6d131a6a-df90-4da8-b07a-5fcc5ef89256'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:28:15.664231 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6d131a6a-df90-4da8-b07a-5fcc5ef89256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57b2cd6b94be49228fec6c9f92d1ccc1', 'request_id': 'req-6d131a6a-df90-4da8-b07a-5fcc5ef89256'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:28:15.670846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6d131a6a-df90-4da8-b07a-5fcc5ef89256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57b2cd6b94be49228fec6c9f92d1ccc1', 'request_id': 'req-6d131a6a-df90-4da8-b07a-5fcc5ef89256'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:28:15.758188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6d131a6a-df90-4da8-b07a-5fcc5ef89256 admin admin] The token cache is being invalidate because project 57b2cd6b94be49228fec6c9f92d1ccc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:28:15.759101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6d131a6a-df90-4da8-b07a-5fcc5ef89256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:28:15.759643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6d131a6a-df90-4da8-b07a-5fcc5ef89256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:28:15.761701 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1148/2274] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:28:15 2022] DELETE /identity/v3/projects/57b2cd6b94be49228fec6c9f92d1ccc1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:28:20.803277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bb874-50a2-4b9d-9934-6e88a6b56161 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:28:20.804788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c47bb874-50a2-4b9d-9934-6e88a6b56161 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:28:20.891484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bb874-50a2-4b9d-9934-6e88a6b56161 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:28:20.975195 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079872f-978f-4b91-8a4d-46680b7828d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:28:20.975988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079872f-978f-4b91-8a4d-46680b7828d6 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:20.976459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079872f-978f-4b91-8a4d-46680b7828d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:20.976867 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079872f-978f-4b91-8a4d-46680b7828d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:20.979028 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a079872f-978f-4b91-8a4d-46680b7828d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:28:21.061339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079872f-978f-4b91-8a4d-46680b7828d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:28:21.063752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079872f-978f-4b91-8a4d-46680b7828d6 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:28:21.078176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1127/2275] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:28:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:28:25.753270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:28:31.071905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:28:38.155936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d131a6a-df90-4da8-b07a-5fcc5ef89256 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:28:38.158247 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6d131a6a-df90-4da8-b07a-5fcc5ef89256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:28:38.177563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d131a6a-df90-4da8-b07a-5fcc5ef89256 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:28:38.183823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462f0d7-75fa-447d-a147-dbbff5bfc312 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:28:38.184493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462f0d7-75fa-447d-a147-dbbff5bfc312 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:38.184780 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462f0d7-75fa-447d-a147-dbbff5bfc312 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:38.185089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462f0d7-75fa-447d-a147-dbbff5bfc312 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:38.187019 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b462f0d7-75fa-447d-a147-dbbff5bfc312 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:28:38.263031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462f0d7-75fa-447d-a147-dbbff5bfc312 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:28:38.264664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462f0d7-75fa-447d-a147-dbbff5bfc312 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:28:38.276917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1149/2276] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:28:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:28:48.271832 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=28 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:28:54.622367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079872f-978f-4b91-8a4d-46680b7828d6 service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:28:54.629315 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a079872f-978f-4b91-8a4d-46680b7828d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:28:54.646542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079872f-978f-4b91-8a4d-46680b7828d6 service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:28:54.655357 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1c888-8ee0-440f-a396-2eb902832e52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:28:54.656041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1c888-8ee0-440f-a396-2eb902832e52 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:28:54.656328 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1c888-8ee0-440f-a396-2eb902832e52 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:28:54.656610 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1c888-8ee0-440f-a396-2eb902832e52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:28:54.658968 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-04e1c888-8ee0-440f-a396-2eb902832e52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:28:54.741181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1c888-8ee0-440f-a396-2eb902832e52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:28:54.742984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1c888-8ee0-440f-a396-2eb902832e52 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:28:54.755151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1128/2277] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:28:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:29:04.749045 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=28 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:29:30.636203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462f0d7-75fa-447d-a147-dbbff5bfc312 service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:29:30.674269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462f0d7-75fa-447d-a147-dbbff5bfc312 service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:29:30.676003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c8bb8-b5fd-4d8a-a637-bf918a435c10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:29:30.676699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c8bb8-b5fd-4d8a-a637-bf918a435c10 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:30.677017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c8bb8-b5fd-4d8a-a637-bf918a435c10 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:30.677363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c8bb8-b5fd-4d8a-a637-bf918a435c10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:30.679480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e90c8bb8-b5fd-4d8a-a637-bf918a435c10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:30.790474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c8bb8-b5fd-4d8a-a637-bf918a435c10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:29:30.793636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c8bb8-b5fd-4d8a-a637-bf918a435c10 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:29:30.812072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1150/2278] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:29:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:29:40.804606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=28 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:29:51.381634 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9649a-1243-402b-9097-61c2489e4209 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:29:51.382520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9649a-1243-402b-9097-61c2489e4209 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:29:51.382718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9649a-1243-402b-9097-61c2489e4209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:29:51.404351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-57b9649a-1243-402b-9097-61c2489e4209 None None] MFA Rules not processed for user `5090fa81be4d41bbb62da2c4f273d537`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:29:51.405663 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-57b9649a-1243-402b-9097-61c2489e4209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:29:51.414081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1129/2279] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:29:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:30:01.398051 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:30:43.701689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c8bb8-b5fd-4d8a-a637-bf918a435c10 service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:30:43.723654 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c8bb8-b5fd-4d8a-a637-bf918a435c10 service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:30:43.734220 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c30641-cd64-4106-a747-2a746b81aa74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:30:43.735377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c30641-cd64-4106-a747-2a746b81aa74 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:30:43.735878 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c30641-cd64-4106-a747-2a746b81aa74 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:30:43.736332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c30641-cd64-4106-a747-2a746b81aa74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:30:43.739334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-86c30641-cd64-4106-a747-2a746b81aa74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:30:43.841952 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c30641-cd64-4106-a747-2a746b81aa74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:30:43.846507 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c30641-cd64-4106-a747-2a746b81aa74 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:30:43.864000 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1151/2280] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:30:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:30:53.855540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=28 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:30:53.900306 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9649a-1243-402b-9097-61c2489e4209 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:30:53.901868 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-57b9649a-1243-402b-9097-61c2489e4209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:30:53.984109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9649a-1243-402b-9097-61c2489e4209 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:30:53.990240 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce6474-10f3-4a26-9e7a-8494fdb3eb23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:30:53.990904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce6474-10f3-4a26-9e7a-8494fdb3eb23 service placement] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:30:53.991188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce6474-10f3-4a26-9e7a-8494fdb3eb23 service placement] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:30:53.991467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce6474-10f3-4a26-9e7a-8494fdb3eb23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:30:53.993579 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-99ce6474-10f3-4a26-9e7a-8494fdb3eb23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:30:54.007534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce6474-10f3-4a26-9e7a-8494fdb3eb23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:30:54.009376 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce6474-10f3-4a26-9e7a-8494fdb3eb23 service placement] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:30:54.021929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1130/2281] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:30:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:31:04.016992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=32 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:29.038073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c30641-cd64-4106-a747-2a746b81aa74 service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:29.039912 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-86c30641-cd64-4106-a747-2a746b81aa74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:29.106747 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c30641-cd64-4106-a747-2a746b81aa74 service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:29.110990 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ec0c6-456a-4b10-bf07-92f419b41375 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:29.111655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ec0c6-456a-4b10-bf07-92f419b41375 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:29.111939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ec0c6-456a-4b10-bf07-92f419b41375 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:29.112215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ec0c6-456a-4b10-bf07-92f419b41375 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:29.114147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-393ec0c6-456a-4b10-bf07-92f419b41375 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:29.190902 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ec0c6-456a-4b10-bf07-92f419b41375 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:29.192386 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ec0c6-456a-4b10-bf07-92f419b41375 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:29.203098 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1152/2282] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:39.198079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=48 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:47.283924 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce6474-10f3-4a26-9e7a-8494fdb3eb23 service placement] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:47.285451 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-99ce6474-10f3-4a26-9e7a-8494fdb3eb23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:47.351560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce6474-10f3-4a26-9e7a-8494fdb3eb23 service placement] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:47.355615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57330f71-67a9-492b-b24a-57afda2f48de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:47.356274 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57330f71-67a9-492b-b24a-57afda2f48de service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:47.356557 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57330f71-67a9-492b-b24a-57afda2f48de service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:47.356842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57330f71-67a9-492b-b24a-57afda2f48de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:47.358956 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-57330f71-67a9-492b-b24a-57afda2f48de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:47.375210 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57330f71-67a9-492b-b24a-57afda2f48de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:47.376727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57330f71-67a9-492b-b24a-57afda2f48de service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:47.392270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1131/2283] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:47.510861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ec0c6-456a-4b10-bf07-92f419b41375 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:31:47.513340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-393ec0c6-456a-4b10-bf07-92f419b41375 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:47.535840 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ec0c6-456a-4b10-bf07-92f419b41375 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:31:47.547085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f735a7-3a12-475b-89b7-1343a09ba0d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:31:47.548048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f735a7-3a12-475b-89b7-1343a09ba0d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:31:47.548704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f735a7-3a12-475b-89b7-1343a09ba0d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:31:47.549149 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f735a7-3a12-475b-89b7-1343a09ba0d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:31:47.552360 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f7f735a7-3a12-475b-89b7-1343a09ba0d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:31:47.575988 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f735a7-3a12-475b-89b7-1343a09ba0d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:31:47.579117 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f735a7-3a12-475b-89b7-1343a09ba0d2 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:31:47.597468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1153/2284] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:31:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:31:57.383789 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=26 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:31:57.588177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=6 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:04.356617 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57330f71-67a9-492b-b24a-57afda2f48de service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:04.373352 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57330f71-67a9-492b-b24a-57afda2f48de service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:04.379416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011350e-f380-4991-b723-8a437388d172 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:04.380078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011350e-f380-4991-b723-8a437388d172 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:04.380364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011350e-f380-4991-b723-8a437388d172 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:04.380648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011350e-f380-4991-b723-8a437388d172 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:04.382775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4011350e-f380-4991-b723-8a437388d172 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:04.472485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011350e-f380-4991-b723-8a437388d172 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:04.474224 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011350e-f380-4991-b723-8a437388d172 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:04.497819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1132/2285] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:32:14.495196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=41 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:32:53.563707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f735a7-3a12-475b-89b7-1343a09ba0d2 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:32:53.576748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f735a7-3a12-475b-89b7-1343a09ba0d2 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:32:53.580745 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb262fc3-c9e6-4a72-8ca2-532aeea652b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:32:53.581433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb262fc3-c9e6-4a72-8ca2-532aeea652b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:32:53.581717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb262fc3-c9e6-4a72-8ca2-532aeea652b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:32:53.581994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb262fc3-c9e6-4a72-8ca2-532aeea652b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:32:53.584145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-bb262fc3-c9e6-4a72-8ca2-532aeea652b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:32:53.595256 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb262fc3-c9e6-4a72-8ca2-532aeea652b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:32:53.596757 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb262fc3-c9e6-4a72-8ca2-532aeea652b1 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:32:53.608013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1154/2286] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:32:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:33:03.607592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=6 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:44.386994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011350e-f380-4991-b723-8a437388d172 service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:44.391114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4011350e-f380-4991-b723-8a437388d172 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:33:44.415451 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011350e-f380-4991-b723-8a437388d172 service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:44.422653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249af7b-91d9-4a65-9f5a-5ee05f0d2c14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:44.423788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249af7b-91d9-4a65-9f5a-5ee05f0d2c14 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:44.424292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249af7b-91d9-4a65-9f5a-5ee05f0d2c14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:44.424796 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f249af7b-91d9-4a65-9f5a-5ee05f0d2c14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:44.436107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249af7b-91d9-4a65-9f5a-5ee05f0d2c14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:44.437659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f249af7b-91d9-4a65-9f5a-5ee05f0d2c14 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:44.471415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1133/2287] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:33:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:33:44.563974 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb262fc3-c9e6-4a72-8ca2-532aeea652b1 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:44.577261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb262fc3-c9e6-4a72-8ca2-532aeea652b1 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:44.583123 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f13ca0-812a-4ab7-b8ae-2f9042cdf81e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:44.584059 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f13ca0-812a-4ab7-b8ae-2f9042cdf81e service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:44.584484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f13ca0-812a-4ab7-b8ae-2f9042cdf81e service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:44.584804 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f13ca0-812a-4ab7-b8ae-2f9042cdf81e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:44.596642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f13ca0-812a-4ab7-b8ae-2f9042cdf81e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:44.598241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f13ca0-812a-4ab7-b8ae-2f9042cdf81e service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:44.632998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1155/2288] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:33:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:33:54.468290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=13 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:54.629789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=13 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:33:54.708830 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249af7b-91d9-4a65-9f5a-5ee05f0d2c14 service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:33:54.762458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f249af7b-91d9-4a65-9f5a-5ee05f0d2c14 service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:33:54.768418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfd0cd-ab45-4924-9f97-01dc00ce3e68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:33:54.769150 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfd0cd-ab45-4924-9f97-01dc00ce3e68 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:33:54.769430 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfd0cd-ab45-4924-9f97-01dc00ce3e68 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:33:54.769699 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfd0cd-ab45-4924-9f97-01dc00ce3e68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:33:54.781559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfd0cd-ab45-4924-9f97-01dc00ce3e68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:33:54.783269 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfd0cd-ab45-4924-9f97-01dc00ce3e68 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:33:54.815941 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1134/2289] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:33:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:34:04.812643 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=27 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:34:30.907201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f13ca0-812a-4ab7-b8ae-2f9042cdf81e service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:34:30.955995 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f13ca0-812a-4ab7-b8ae-2f9042cdf81e service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:34:30.961622 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a5d4f-606e-4222-8ca1-f5f0012a2172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:34:30.962646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a5d4f-606e-4222-8ca1-f5f0012a2172 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:34:30.963297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a5d4f-606e-4222-8ca1-f5f0012a2172 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:34:30.963583 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a5d4f-606e-4222-8ca1-f5f0012a2172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:34:30.974652 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a5d4f-606e-4222-8ca1-f5f0012a2172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:34:30.976144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8a5d4f-606e-4222-8ca1-f5f0012a2172 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:34:31.009556 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1156/2290] 10.222.0.60 () {60 vars in 1396 bytes} [Wed Jun 15 10:34:30 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3238 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:34:41.006289 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=25 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:35:56.885203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfd0cd-ab45-4924-9f97-01dc00ce3e68 service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:35:56.886584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-5ddfd0cd-ab45-4924-9f97-01dc00ce3e68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:35:56.935049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfd0cd-ab45-4924-9f97-01dc00ce3e68 service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:35:56.940153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d50a7-b4f3-43a7-b61a-24ce4a28a743 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:35:56.940802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d50a7-b4f3-43a7-b61a-24ce4a28a743 service placement] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:35:56.941123 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d50a7-b4f3-43a7-b61a-24ce4a28a743 service placement] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:35:56.941412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d50a7-b4f3-43a7-b61a-24ce4a28a743 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:35:56.950302 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d50a7-b4f3-43a7-b61a-24ce4a28a743 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:35:56.951894 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d50a7-b4f3-43a7-b61a-24ce4a28a743 service placement] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:35:56.962710 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1135/2291] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:35:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:36:06.958199 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=19 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:29.380912 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a5d4f-606e-4222-8ca1-f5f0012a2172 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:29.401398 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a5d4f-606e-4222-8ca1-f5f0012a2172 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:29.405103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9356655-be50-4846-b10d-13b489022d8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:29.405689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9356655-be50-4846-b10d-13b489022d8b service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:29.405973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9356655-be50-4846-b10d-13b489022d8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:29.406251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9356655-be50-4846-b10d-13b489022d8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:29.415073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9356655-be50-4846-b10d-13b489022d8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:29.416509 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9356655-be50-4846-b10d-13b489022d8b service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:29.444348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1157/2292] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:39.441204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=17 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:45.037304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d50a7-b4f3-43a7-b61a-24ce4a28a743 service placement] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:45.047835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d50a7-b4f3-43a7-b61a-24ce4a28a743 service placement] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:45.051563 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67b2e0-0e6e-46d7-96dd-3e06cb5cf358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:45.052206 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67b2e0-0e6e-46d7-96dd-3e06cb5cf358 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:45.052483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67b2e0-0e6e-46d7-96dd-3e06cb5cf358 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:45.052761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67b2e0-0e6e-46d7-96dd-3e06cb5cf358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:45.070756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67b2e0-0e6e-46d7-96dd-3e06cb5cf358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:45.072184 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67b2e0-0e6e-46d7-96dd-3e06cb5cf358 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:45.094781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1136/2293] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:37:45.206318 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9356655-be50-4846-b10d-13b489022d8b service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:45.216291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9356655-be50-4846-b10d-13b489022d8b service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:45.219956 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825dafd-70c1-4c1c-be9a-f9582fb88129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:45.220596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825dafd-70c1-4c1c-be9a-f9582fb88129 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:45.220877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825dafd-70c1-4c1c-be9a-f9582fb88129 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:45.221187 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825dafd-70c1-4c1c-be9a-f9582fb88129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:45.232901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825dafd-70c1-4c1c-be9a-f9582fb88129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:45.234428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825dafd-70c1-4c1c-be9a-f9582fb88129 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:45.245881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1158/2294] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:45.456030 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67b2e0-0e6e-46d7-96dd-3e06cb5cf358 service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:45.467688 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67b2e0-0e6e-46d7-96dd-3e06cb5cf358 service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:45.473521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c8661-c5d4-46c3-adef-c9eb21eb42a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:45.474192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c8661-c5d4-46c3-adef-c9eb21eb42a2 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:45.474478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c8661-c5d4-46c3-adef-c9eb21eb42a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:45.474761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c8661-c5d4-46c3-adef-c9eb21eb42a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:45.483990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c8661-c5d4-46c3-adef-c9eb21eb42a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:45.486496 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c8661-c5d4-46c3-adef-c9eb21eb42a2 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:45.495492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1137/2295] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:45.548208 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825dafd-70c1-4c1c-be9a-f9582fb88129 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:45.559437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825dafd-70c1-4c1c-be9a-f9582fb88129 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:45.564942 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14159004-11c9-4f0f-bfe9-9624c496a46d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:45.565604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14159004-11c9-4f0f-bfe9-9624c496a46d service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:45.565890 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14159004-11c9-4f0f-bfe9-9624c496a46d service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:45.566166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14159004-11c9-4f0f-bfe9-9624c496a46d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:45.574810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14159004-11c9-4f0f-bfe9-9624c496a46d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:45.576401 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14159004-11c9-4f0f-bfe9-9624c496a46d service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:45.603923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1159/2296] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:55.492091 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=19 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:37:55.554819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c8661-c5d4-46c3-adef-c9eb21eb42a2 service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:37:55.565111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c8661-c5d4-46c3-adef-c9eb21eb42a2 service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:37:55.568742 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30dfd0a-9507-4052-b339-80b98e03e793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:37:55.569418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30dfd0a-9507-4052-b339-80b98e03e793 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:37:55.575219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30dfd0a-9507-4052-b339-80b98e03e793 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:37:55.575219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30dfd0a-9507-4052-b339-80b98e03e793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:37:55.578039 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30dfd0a-9507-4052-b339-80b98e03e793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:37:55.579498 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30dfd0a-9507-4052-b339-80b98e03e793 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:37:55.587279 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1138/2297] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:37:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:37:55.601215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=19 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:38:05.694035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=6 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:45.432252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14159004-11c9-4f0f-bfe9-9624c496a46d service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:45.442560 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14159004-11c9-4f0f-bfe9-9624c496a46d service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:45.446349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd921e-a408-4bed-8f45-47cce0e73cfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:45.446992 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd921e-a408-4bed-8f45-47cce0e73cfb service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:45.447274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd921e-a408-4bed-8f45-47cce0e73cfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:45.447552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd921e-a408-4bed-8f45-47cce0e73cfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:45.449521 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-00dd921e-a408-4bed-8f45-47cce0e73cfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:45.524886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd921e-a408-4bed-8f45-47cce0e73cfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:45.526353 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd921e-a408-4bed-8f45-47cce0e73cfb service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:45.536319 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1160/2298] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:39:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:39:45.587971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30dfd0a-9507-4052-b339-80b98e03e793 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:39:45.598166 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30dfd0a-9507-4052-b339-80b98e03e793 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:39:45.601830 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658ead0-9b52-4396-b492-83e2deb68938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:39:45.602466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658ead0-9b52-4396-b492-83e2deb68938 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:39:45.602754 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658ead0-9b52-4396-b492-83e2deb68938 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:39:45.603036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658ead0-9b52-4396-b492-83e2deb68938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:39:45.605091 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-2658ead0-9b52-4396-b492-83e2deb68938 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:39:45.681449 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658ead0-9b52-4396-b492-83e2deb68938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:39:45.682808 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658ead0-9b52-4396-b492-83e2deb68938 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:39:45.692466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1139/2299] 10.222.0.60 () {60 vars in 1396 bytes} [Wed Jun 15 10:39:45 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:39:55.532002 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=33 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:39:55.687971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=33 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:41:02.841244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd921e-a408-4bed-8f45-47cce0e73cfb service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:02.842818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-00dd921e-a408-4bed-8f45-47cce0e73cfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:02.917786 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd921e-a408-4bed-8f45-47cce0e73cfb service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:02.923484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5763f0a-bb73-47a9-ad29-4a72092db228 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:02.924139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5763f0a-bb73-47a9-ad29-4a72092db228 service placement] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:02.924427 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5763f0a-bb73-47a9-ad29-4a72092db228 service placement] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:02.924707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5763f0a-bb73-47a9-ad29-4a72092db228 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:02.926702 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e5763f0a-bb73-47a9-ad29-4a72092db228 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:02.940250 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5763f0a-bb73-47a9-ad29-4a72092db228 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:02.942041 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5763f0a-bb73-47a9-ad29-4a72092db228 service placement] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:02.952610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1161/2300] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:41:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:41:12.948053 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=32 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:41:45.959425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658ead0-9b52-4396-b492-83e2deb68938 service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:45.962870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-2658ead0-9b52-4396-b492-83e2deb68938 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:46.063941 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658ead0-9b52-4396-b492-83e2deb68938 service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:46.069729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e36576-7e69-4dbe-8d01-f94b750ca905 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:46.070404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e36576-7e69-4dbe-8d01-f94b750ca905 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:46.071274 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e36576-7e69-4dbe-8d01-f94b750ca905 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:46.071556 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e36576-7e69-4dbe-8d01-f94b750ca905 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:46.073696 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c4e36576-7e69-4dbe-8d01-f94b750ca905 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:46.179464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e36576-7e69-4dbe-8d01-f94b750ca905 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:46.179706 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5763f0a-bb73-47a9-ad29-4a72092db228 service placement] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:46.180276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e36576-7e69-4dbe-8d01-f94b750ca905 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:46.201296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1140/2301] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:41:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:41:46.254102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e36576-7e69-4dbe-8d01-f94b750ca905 service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:41:46.256669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c4e36576-7e69-4dbe-8d01-f94b750ca905 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:46.272295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5763f0a-bb73-47a9-ad29-4a72092db228 service placement] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:46.275642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e36576-7e69-4dbe-8d01-f94b750ca905 service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:41:46.283257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73eeb3-57b2-4e6e-87d1-8f0230883b94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:46.284178 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73eeb3-57b2-4e6e-87d1-8f0230883b94 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:46.284531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73eeb3-57b2-4e6e-87d1-8f0230883b94 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:46.284834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73eeb3-57b2-4e6e-87d1-8f0230883b94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:46.286612 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e869be0-4e56-4aa4-8717-eb785ba55888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:41:46.287254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ef73eeb3-57b2-4e6e-87d1-8f0230883b94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:41:46.290599 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e869be0-4e56-4aa4-8717-eb785ba55888 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:41:46.291485 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e869be0-4e56-4aa4-8717-eb785ba55888 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:41:46.292710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e869be0-4e56-4aa4-8717-eb785ba55888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:41:46.316647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e869be0-4e56-4aa4-8717-eb785ba55888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:46.319969 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e869be0-4e56-4aa4-8717-eb785ba55888 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:46.340110 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1162/2302] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:41:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:41:46.419609 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73eeb3-57b2-4e6e-87d1-8f0230883b94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:41:46.421980 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73eeb3-57b2-4e6e-87d1-8f0230883b94 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:41:46.437311 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1141/2303] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:41:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:41:56.330678 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=26 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:41:56.427998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:43:03.512332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e869be0-4e56-4aa4-8717-eb785ba55888 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:43:03.513980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-1e869be0-4e56-4aa4-8717-eb785ba55888 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:43:03.591658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e869be0-4e56-4aa4-8717-eb785ba55888 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:43:03.595221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fb02c-ec3d-4f65-955e-4f5430f1001b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:43:03.595863 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fb02c-ec3d-4f65-955e-4f5430f1001b service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:43:03.596148 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fb02c-ec3d-4f65-955e-4f5430f1001b service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:43:03.596424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fb02c-ec3d-4f65-955e-4f5430f1001b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:43:03.598326 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-a47fb02c-ec3d-4f65-955e-4f5430f1001b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:43:03.609252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fb02c-ec3d-4f65-955e-4f5430f1001b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:43:03.612118 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fb02c-ec3d-4f65-955e-4f5430f1001b service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:43:03.618970 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1163/2304] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:43:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:43:13.615257 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=31 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:43:29.629894 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73eeb3-57b2-4e6e-87d1-8f0230883b94 service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:43:29.638474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ef73eeb3-57b2-4e6e-87d1-8f0230883b94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:43:29.658336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73eeb3-57b2-4e6e-87d1-8f0230883b94 service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:43:29.663049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1c21a-edda-41fb-8efd-85a5bfd3fbfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:43:29.663737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1c21a-edda-41fb-8efd-85a5bfd3fbfb service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:43:29.664105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1c21a-edda-41fb-8efd-85a5bfd3fbfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:43:29.664392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1c21a-edda-41fb-8efd-85a5bfd3fbfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:43:29.666413 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-eba1c21a-edda-41fb-8efd-85a5bfd3fbfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:43:29.744806 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1c21a-edda-41fb-8efd-85a5bfd3fbfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:43:29.746276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba1c21a-edda-41fb-8efd-85a5bfd3fbfb service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:43:29.755950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1142/2305] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:43:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:43:39.751196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=38 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:43:46.404130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fb02c-ec3d-4f65-955e-4f5430f1001b service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:43:46.405907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-a47fb02c-ec3d-4f65-955e-4f5430f1001b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:43:46.418989 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fb02c-ec3d-4f65-955e-4f5430f1001b service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:43:46.422706 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351b5f4-c9ec-458f-a50b-25fb72bb6dbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:43:46.423358 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351b5f4-c9ec-458f-a50b-25fb72bb6dbe service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:43:46.423644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351b5f4-c9ec-458f-a50b-25fb72bb6dbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:43:46.423932 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351b5f4-c9ec-458f-a50b-25fb72bb6dbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:43:46.425924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c351b5f4-c9ec-458f-a50b-25fb72bb6dbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:43:46.441269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c351b5f4-c9ec-458f-a50b-25fb72bb6dbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:43:46.442777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c351b5f4-c9ec-458f-a50b-25fb72bb6dbe service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:43:46.454395 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1164/2306] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:43:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:43:46.509586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1c21a-edda-41fb-8efd-85a5bfd3fbfb service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:43:46.518868 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba1c21a-edda-41fb-8efd-85a5bfd3fbfb service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:43:46.523654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404f72e-8fc4-4944-bad1-c97220682628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:43:46.524251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404f72e-8fc4-4944-bad1-c97220682628 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:43:46.524532 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404f72e-8fc4-4944-bad1-c97220682628 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:43:46.524806 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404f72e-8fc4-4944-bad1-c97220682628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:43:46.526846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e404f72e-8fc4-4944-bad1-c97220682628 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:43:46.542524 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404f72e-8fc4-4944-bad1-c97220682628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:43:46.544027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404f72e-8fc4-4944-bad1-c97220682628 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:43:46.557450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1143/2307] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:43:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:43:46.565484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351b5f4-c9ec-458f-a50b-25fb72bb6dbe service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:43:46.574714 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351b5f4-c9ec-458f-a50b-25fb72bb6dbe service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:43:46.578486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40083b-6202-41eb-b1d1-b19bea1c1139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:43:46.579138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40083b-6202-41eb-b1d1-b19bea1c1139 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:43:46.579443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40083b-6202-41eb-b1d1-b19bea1c1139 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:43:46.579723 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40083b-6202-41eb-b1d1-b19bea1c1139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:43:46.581731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5a40083b-6202-41eb-b1d1-b19bea1c1139 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:43:46.662519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40083b-6202-41eb-b1d1-b19bea1c1139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:43:46.664205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a40083b-6202-41eb-b1d1-b19bea1c1139 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:43:46.674831 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1165/2308] 10.222.0.60 () {60 vars in 1396 bytes} [Wed Jun 15 10:43:46 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:43:46.776531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404f72e-8fc4-4944-bad1-c97220682628 service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:43:46.785781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404f72e-8fc4-4944-bad1-c97220682628 service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:43:46.790536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e4aa3-ef84-4fb3-9bdc-e533480670da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:43:46.791176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e4aa3-ef84-4fb3-9bdc-e533480670da service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:43:46.791455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e4aa3-ef84-4fb3-9bdc-e533480670da service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:43:46.791729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e4aa3-ef84-4fb3-9bdc-e533480670da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:43:46.793647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c65e4aa3-ef84-4fb3-9bdc-e533480670da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:43:46.880724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e4aa3-ef84-4fb3-9bdc-e533480670da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:43:46.882322 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e4aa3-ef84-4fb3-9bdc-e533480670da service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:43:46.901860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1144/2309] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:43:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:43:56.669799 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=39 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:43:56.899180 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=47 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:45:29.792729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40083b-6202-41eb-b1d1-b19bea1c1139 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:45:29.810930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40083b-6202-41eb-b1d1-b19bea1c1139 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:45:29.815293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f78b3a-a4ac-41c5-908d-567d09fb5a92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:45:29.815931 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f78b3a-a4ac-41c5-908d-567d09fb5a92 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:45:29.816213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f78b3a-a4ac-41c5-908d-567d09fb5a92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:45:29.816489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f78b3a-a4ac-41c5-908d-567d09fb5a92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:45:29.842009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f78b3a-a4ac-41c5-908d-567d09fb5a92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:45:29.843455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f78b3a-a4ac-41c5-908d-567d09fb5a92 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:45:29.871006 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1166/2310] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:45:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:45:29.964684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e4aa3-ef84-4fb3-9bdc-e533480670da service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:45:29.984989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e4aa3-ef84-4fb3-9bdc-e533480670da service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:45:29.989739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0f5d5-2a41-4bb5-a301-d2518311f3a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:45:29.990401 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0f5d5-2a41-4bb5-a301-d2518311f3a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:45:29.990684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0f5d5-2a41-4bb5-a301-d2518311f3a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:45:29.990958 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0f5d5-2a41-4bb5-a301-d2518311f3a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:45:30.000428 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0f5d5-2a41-4bb5-a301-d2518311f3a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:45:30.002474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0f5d5-2a41-4bb5-a301-d2518311f3a4 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:45:30.030291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1145/2311] 10.222.0.60 () {60 vars in 1396 bytes} [Wed Jun 15 10:45:29 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:45:33.026862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f6a8f-6522-4817-bd24-9aa1d6fd1ee3 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:45:33.034092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f6a8f-6522-4817-bd24-9aa1d6fd1ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:45:33.034092 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f6a8f-6522-4817-bd24-9aa1d6fd1ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:45:33.064888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-a77f6a8f-6522-4817-bd24-9aa1d6fd1ee3 None None] MFA Rules not processed for user `0ac3d9b2893b4f078c2a379696aaff34`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:45:33.066174 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-a77f6a8f-6522-4817-bd24-9aa1d6fd1ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:45:33.074567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1167/2312] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:45:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:45:33.569172 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0f5d5-2a41-4bb5-a301-d2518311f3a4 service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:45:33.598994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0f5d5-2a41-4bb5-a301-d2518311f3a4 service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:45:33.602997 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bfd99-76d9-453c-b57b-4e537d2963cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:45:33.603685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bfd99-76d9-453c-b57b-4e537d2963cb service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:45:33.603974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bfd99-76d9-453c-b57b-4e537d2963cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:45:33.604401 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bfd99-76d9-453c-b57b-4e537d2963cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:45:33.606257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-698bfd99-76d9-453c-b57b-4e537d2963cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:45:33.616583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bfd99-76d9-453c-b57b-4e537d2963cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:45:33.618058 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bfd99-76d9-453c-b57b-4e537d2963cb service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:45:33.626121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1146/2313] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:45:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:45:34.353817 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f6a8f-6522-4817-bd24-9aa1d6fd1ee3 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:45:34.380069 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f6a8f-6522-4817-bd24-9aa1d6fd1ee3 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:45:34.391314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb7350-2889-43e5-b8ce-de01ae417dc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:45:34.392205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb7350-2889-43e5-b8ce-de01ae417dc9 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:45:34.392652 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb7350-2889-43e5-b8ce-de01ae417dc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:45:34.393098 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb7350-2889-43e5-b8ce-de01ae417dc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:45:34.395224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-eccb7350-2889-43e5-b8ce-de01ae417dc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:45:34.410028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb7350-2889-43e5-b8ce-de01ae417dc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:45:34.411887 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb7350-2889-43e5-b8ce-de01ae417dc9 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:45:34.423729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1168/2314] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:45:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:45:43.622534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=31 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:45:44.418773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=39 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:46:07.868352 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bfd99-76d9-453c-b57b-4e537d2963cb service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:46:07.892406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bfd99-76d9-453c-b57b-4e537d2963cb service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:46:07.897686 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e72972-c34d-41f5-ae0a-8b677d715ebb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:46:07.898351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e72972-c34d-41f5-ae0a-8b677d715ebb service placement] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:46:07.898636 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e72972-c34d-41f5-ae0a-8b677d715ebb service placement] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:46:07.898911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e72972-c34d-41f5-ae0a-8b677d715ebb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:46:07.909804 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e72972-c34d-41f5-ae0a-8b677d715ebb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:46:07.911444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e72972-c34d-41f5-ae0a-8b677d715ebb service placement] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:46:07.923549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1147/2315] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:46:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:46:17.918198 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=10 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:46:46.591730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb7350-2889-43e5-b8ce-de01ae417dc9 service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:46:46.603526 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb7350-2889-43e5-b8ce-de01ae417dc9 service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:46:46.609388 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b56611-ea38-459e-8374-3356788631b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:46:46.610031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b56611-ea38-459e-8374-3356788631b0 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:46:46.610319 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b56611-ea38-459e-8374-3356788631b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:46:46.610597 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b56611-ea38-459e-8374-3356788631b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:46:46.619583 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b56611-ea38-459e-8374-3356788631b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:46:46.621267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b56611-ea38-459e-8374-3356788631b0 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:46:46.655087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1169/2316] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:46:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:46:56.651948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=13 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:47:36.259970 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e72972-c34d-41f5-ae0a-8b677d715ebb service placement] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:47:36.280760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e72972-c34d-41f5-ae0a-8b677d715ebb service placement] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:47:36.284506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da39418-9ccb-41ff-a835-3941832fcb78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:47:36.285192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da39418-9ccb-41ff-a835-3941832fcb78 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:47:36.285478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da39418-9ccb-41ff-a835-3941832fcb78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:47:36.285759 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da39418-9ccb-41ff-a835-3941832fcb78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:47:36.295998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da39418-9ccb-41ff-a835-3941832fcb78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:47:36.297514 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da39418-9ccb-41ff-a835-3941832fcb78 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:47:36.328643 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1148/2317] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:47:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:47:46.324623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=17 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:48:08.495536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b56611-ea38-459e-8374-3356788631b0 service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:48:08.505029 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b56611-ea38-459e-8374-3356788631b0 service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:48:08.508567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84d9c7-c8b5-4923-81d2-51c1c750571d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:48:08.509308 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84d9c7-c8b5-4923-81d2-51c1c750571d service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:48:08.509591 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84d9c7-c8b5-4923-81d2-51c1c750571d service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:48:08.509865 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84d9c7-c8b5-4923-81d2-51c1c750571d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:48:08.517754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b84d9c7-c8b5-4923-81d2-51c1c750571d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:48:08.519195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b84d9c7-c8b5-4923-81d2-51c1c750571d service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:48:08.527531 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1170/2318] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:48:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:48:18.523774 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=6 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:48:45.550707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da39418-9ccb-41ff-a835-3941832fcb78 service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:48:45.561467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da39418-9ccb-41ff-a835-3941832fcb78 service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:48:45.565077 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de60e05-2329-46c9-a82e-9e31df8b3782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:48:45.565710 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de60e05-2329-46c9-a82e-9e31df8b3782 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:48:45.565990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de60e05-2329-46c9-a82e-9e31df8b3782 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:48:45.566637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de60e05-2329-46c9-a82e-9e31df8b3782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:48:45.584213 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de60e05-2329-46c9-a82e-9e31df8b3782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:48:45.585677 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de60e05-2329-46c9-a82e-9e31df8b3782 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:48:45.609473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1149/2319] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:48:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:48:46.719204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84d9c7-c8b5-4923-81d2-51c1c750571d service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:48:46.720902 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6b84d9c7-c8b5-4923-81d2-51c1c750571d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:48:46.744509 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84d9c7-c8b5-4923-81d2-51c1c750571d service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:48:46.748264 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787da9c8-e7fb-4ebb-b4bf-ada65166104a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:48:46.748923 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787da9c8-e7fb-4ebb-b4bf-ada65166104a service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:48:46.749271 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787da9c8-e7fb-4ebb-b4bf-ada65166104a service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:48:46.749522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787da9c8-e7fb-4ebb-b4bf-ada65166104a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:48:46.761188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787da9c8-e7fb-4ebb-b4bf-ada65166104a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:48:46.762662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787da9c8-e7fb-4ebb-b4bf-ada65166104a service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:48:46.775213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1171/2320] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:48:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:48:55.606794 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=13 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:48:56.769379 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=10 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:49:36.752141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de60e05-2329-46c9-a82e-9e31df8b3782 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:49:36.761466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de60e05-2329-46c9-a82e-9e31df8b3782 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:49:36.765027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21e99a-b46d-4ab5-91f2-aa46ffce3aa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:49:36.765699 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21e99a-b46d-4ab5-91f2-aa46ffce3aa4 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:49:36.765978 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21e99a-b46d-4ab5-91f2-aa46ffce3aa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:49:36.766251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21e99a-b46d-4ab5-91f2-aa46ffce3aa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:49:36.775892 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21e99a-b46d-4ab5-91f2-aa46ffce3aa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:49:36.777353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21e99a-b46d-4ab5-91f2-aa46ffce3aa4 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:49:36.787070 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1150/2321] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:49:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:49:36.835604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787da9c8-e7fb-4ebb-b4bf-ada65166104a service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:49:36.845365 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787da9c8-e7fb-4ebb-b4bf-ada65166104a service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:49:36.848981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e06e8-3823-4d18-bbe5-f2bcd9264459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:49:36.849637 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e06e8-3823-4d18-bbe5-f2bcd9264459 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:49:36.849926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e06e8-3823-4d18-bbe5-f2bcd9264459 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:49:36.850204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e06e8-3823-4d18-bbe5-f2bcd9264459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:49:36.859415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e06e8-3823-4d18-bbe5-f2bcd9264459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:49:36.860849 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e06e8-3823-4d18-bbe5-f2bcd9264459 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:49:36.889635 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1172/2322] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:49:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:49:36.940505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21e99a-b46d-4ab5-91f2-aa46ffce3aa4 service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:49:36.949582 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21e99a-b46d-4ab5-91f2-aa46ffce3aa4 service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:49:36.953290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4d93f-00c9-404b-8cae-23e91d70eb81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:49:36.953913 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d93f-00c9-404b-8cae-23e91d70eb81 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:49:36.954194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d93f-00c9-404b-8cae-23e91d70eb81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:49:36.954469 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d93f-00c9-404b-8cae-23e91d70eb81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:49:36.963519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4d93f-00c9-404b-8cae-23e91d70eb81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:49:36.965262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4d93f-00c9-404b-8cae-23e91d70eb81 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:49:36.994140 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1151/2323] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:49:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:49:46.886965 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=13 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:49:46.990415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=19 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:50:30.098694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e06e8-3823-4d18-bbe5-f2bcd9264459 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:50:30.111904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e06e8-3823-4d18-bbe5-f2bcd9264459 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:50:30.117305 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535114e-1c31-4d13-8d49-ac1e90a4639e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:50:30.117946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535114e-1c31-4d13-8d49-ac1e90a4639e service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:50:30.118331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535114e-1c31-4d13-8d49-ac1e90a4639e service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:50:30.118606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535114e-1c31-4d13-8d49-ac1e90a4639e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:50:30.120502 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d535114e-1c31-4d13-8d49-ac1e90a4639e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:50:30.192838 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535114e-1c31-4d13-8d49-ac1e90a4639e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:50:30.194454 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535114e-1c31-4d13-8d49-ac1e90a4639e service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:50:30.203897 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1173/2324] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:50:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:50:30.468921 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4d93f-00c9-404b-8cae-23e91d70eb81 service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:50:30.482358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4d93f-00c9-404b-8cae-23e91d70eb81 service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:50:30.488919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671b9bb-210c-458c-a2b7-0fc075adf670 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:50:30.489583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671b9bb-210c-458c-a2b7-0fc075adf670 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:50:30.489865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671b9bb-210c-458c-a2b7-0fc075adf670 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:50:30.490142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671b9bb-210c-458c-a2b7-0fc075adf670 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:50:30.492057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d671b9bb-210c-458c-a2b7-0fc075adf670 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:50:30.567815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671b9bb-210c-458c-a2b7-0fc075adf670 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:50:30.569491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671b9bb-210c-458c-a2b7-0fc075adf670 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:50:30.579630 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1152/2325] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:50:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:50:40.199495 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=33 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:50:40.574922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=33 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:51:16.824760 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535114e-1c31-4d13-8d49-ac1e90a4639e service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:51:16.827798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d535114e-1c31-4d13-8d49-ac1e90a4639e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:51:16.921617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535114e-1c31-4d13-8d49-ac1e90a4639e service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:51:16.927167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9733c8-e9bb-4eb4-aabe-cc1fbba3ff8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:51:16.927828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9733c8-e9bb-4eb4-aabe-cc1fbba3ff8a service placement] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:51:16.928188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9733c8-e9bb-4eb4-aabe-cc1fbba3ff8a service placement] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:51:16.928576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9733c8-e9bb-4eb4-aabe-cc1fbba3ff8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:51:16.930739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ae9733c8-e9bb-4eb4-aabe-cc1fbba3ff8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:51:16.945266 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9733c8-e9bb-4eb4-aabe-cc1fbba3ff8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:51:16.946869 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9733c8-e9bb-4eb4-aabe-cc1fbba3ff8a service placement] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:51:16.957733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1174/2326] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:51:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:51:26.953163 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=37 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:51:36.071655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671b9bb-210c-458c-a2b7-0fc075adf670 service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:51:36.080994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671b9bb-210c-458c-a2b7-0fc075adf670 service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:51:36.084681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1322e0-32dd-418f-b520-8717078334a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:51:36.085380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1322e0-32dd-418f-b520-8717078334a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:51:36.085666 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1322e0-32dd-418f-b520-8717078334a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:51:36.085946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1322e0-32dd-418f-b520-8717078334a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:51:36.087940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-dd1322e0-32dd-418f-b520-8717078334a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:51:36.174086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1322e0-32dd-418f-b520-8717078334a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:51:36.175440 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1322e0-32dd-418f-b520-8717078334a1 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:51:36.194893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1153/2327] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:51:38.180872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9733c8-e9bb-4eb4-aabe-cc1fbba3ff8a service placement] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:51:38.195151 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9733c8-e9bb-4eb4-aabe-cc1fbba3ff8a service placement] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:51:38.199752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e5586-86df-4e79-8c93-658aab622de7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:51:38.200746 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e5586-86df-4e79-8c93-658aab622de7 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:51:38.201055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e5586-86df-4e79-8c93-658aab622de7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:51:38.201337 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e5586-86df-4e79-8c93-658aab622de7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:51:38.203973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-3a2e5586-86df-4e79-8c93-658aab622de7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:51:38.217744 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e5586-86df-4e79-8c93-658aab622de7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:51:38.222595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e5586-86df-4e79-8c93-658aab622de7 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:51:38.231015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1175/2328] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:51:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:51:40.589049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1322e0-32dd-418f-b520-8717078334a1 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:51:40.591684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-dd1322e0-32dd-418f-b520-8717078334a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:51:40.609360 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1322e0-32dd-418f-b520-8717078334a1 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:51:40.616489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced6eaa-aafb-477a-91a7-1dd52695e65a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:51:40.616489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced6eaa-aafb-477a-91a7-1dd52695e65a service placement] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:51:40.616489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced6eaa-aafb-477a-91a7-1dd52695e65a service placement] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:51:40.623945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced6eaa-aafb-477a-91a7-1dd52695e65a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:51:40.626616 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4ced6eaa-aafb-477a-91a7-1dd52695e65a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:51:40.641963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced6eaa-aafb-477a-91a7-1dd52695e65a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:51:40.644879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced6eaa-aafb-477a-91a7-1dd52695e65a service placement] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:51:40.665012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1154/2329] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:51:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:51:42.952164 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecc2a1-c87c-4a7b-b9c3-6d822e6afe51 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:51:42.952634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecc2a1-c87c-4a7b-b9c3-6d822e6afe51 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:51:42.952634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecc2a1-c87c-4a7b-b9c3-6d822e6afe51 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:51:42.954379 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1176/2330] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jun 15 10:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 10:51:43.187355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0137de-ac0e-4504-b10e-2a005a04835d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:51:43.187919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0137de-ac0e-4504-b10e-2a005a04835d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:51:43.187919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0137de-ac0e-4504-b10e-2a005a04835d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:51:43.252077 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-eb0137de-ac0e-4504-b10e-2a005a04835d None None] MFA Rules not processed for user `12899a7a59b54e3199b277ff94e30d42`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:51:43.253284 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-eb0137de-ac0e-4504-b10e-2a005a04835d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:51:43.262653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1155/2331] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jun 15 10:51:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:51:43.275149 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecc2a1-c87c-4a7b-b9c3-6d822e6afe51 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:51:43.290789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecc2a1-c87c-4a7b-b9c3-6d822e6afe51 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:51:43.299476 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0ed67-85f2-48d2-8362-b49cb74cb89d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:51:43.300173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0ed67-85f2-48d2-8362-b49cb74cb89d service placement] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:51:43.300473 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0ed67-85f2-48d2-8362-b49cb74cb89d service placement] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:51:43.300739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0ed67-85f2-48d2-8362-b49cb74cb89d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:51:43.311154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0ed67-85f2-48d2-8362-b49cb74cb89d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:51:43.312594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0ed67-85f2-48d2-8362-b49cb74cb89d service placement] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:51:43.334759 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1177/2332] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:51:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:51:45.683065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0137de-ac0e-4504-b10e-2a005a04835d None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:51:45.685432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-eb0137de-ac0e-4504-b10e-2a005a04835d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:51:45.702185 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0137de-ac0e-4504-b10e-2a005a04835d None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:51:45.706957 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7db731-7df1-4cde-b159-4c319c46d5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:51:45.707965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7db731-7df1-4cde-b159-4c319c46d5c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:51:45.708298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7db731-7df1-4cde-b159-4c319c46d5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:51:45.708577 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7db731-7df1-4cde-b159-4c319c46d5c6 admin admin] PATH_INFO: `/v3/users/abbf3822f4a6415b964112a383ad5509` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:51:45.710131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7db731-7df1-4cde-b159-4c319c46d5c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abbf3822f4a6415b964112a383ad5509)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:51:45.712935 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7db731-7df1-4cde-b159-4c319c46d5c6 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:51:45.951882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8e7db731-7df1-4cde-b159-4c319c46d5c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abbf3822f4a6415b964112a383ad5509'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:46.091199 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8e7db731-7df1-4cde-b159-4c319c46d5c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abbf3822f4a6415b964112a383ad5509'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:46.091985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8e7db731-7df1-4cde-b159-4c319c46d5c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abbf3822f4a6415b964112a383ad5509'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:46.093044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8e7db731-7df1-4cde-b159-4c319c46d5c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abbf3822f4a6415b964112a383ad5509'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:46.126419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:51:46.128005 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-8e7db731-7df1-4cde-b159-4c319c46d5c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abbf3822f4a6415b964112a383ad5509'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:46.151600 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1156/2333] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:51:45 2022] DELETE /identity/v3/users/abbf3822f4a6415b964112a383ad5509 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:51:46.560793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0ed67-85f2-48d2-8362-b49cb74cb89d service placement] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:51:46.573269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0ed67-85f2-48d2-8362-b49cb74cb89d service placement] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:51:46.639795 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7448b0b-71e2-41ce-b9c8-21873481cc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:51:46.640675 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7448b0b-71e2-41ce-b9c8-21873481cc16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:51:46.640973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7448b0b-71e2-41ce-b9c8-21873481cc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:51:46.641444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7448b0b-71e2-41ce-b9c8-21873481cc16 admin admin] PATH_INFO: `/v3/projects/492be26275404bc3a439ae049bf4b9f4` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:51:46.642919 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7448b0b-71e2-41ce-b9c8-21873481cc16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=492be26275404bc3a439ae049bf4b9f4)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:51:46.644895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7448b0b-71e2-41ce-b9c8-21873481cc16 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:51:46.762465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a7448b0b-71e2-41ce-b9c8-21873481cc16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '492be26275404bc3a439ae049bf4b9f4', 'request_id': 'req-a7448b0b-71e2-41ce-b9c8-21873481cc16'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:46.770844 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a7448b0b-71e2-41ce-b9c8-21873481cc16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '492be26275404bc3a439ae049bf4b9f4', 'request_id': 'req-a7448b0b-71e2-41ce-b9c8-21873481cc16'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:46.781346 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a7448b0b-71e2-41ce-b9c8-21873481cc16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '492be26275404bc3a439ae049bf4b9f4', 'request_id': 'req-a7448b0b-71e2-41ce-b9c8-21873481cc16'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:46.872183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a7448b0b-71e2-41ce-b9c8-21873481cc16 admin admin] The token cache is being invalidate because project 492be26275404bc3a439ae049bf4b9f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:51:46.872477 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a7448b0b-71e2-41ce-b9c8-21873481cc16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:46.873183 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a7448b0b-71e2-41ce-b9c8-21873481cc16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:46.874769 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1178/2334] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:51:46 2022] DELETE /identity/v3/projects/492be26275404bc3a439ae049bf4b9f4 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:51:46.879373 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7db731-7df1-4cde-b159-4c319c46d5c6 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:51:46.880977 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-8e7db731-7df1-4cde-b159-4c319c46d5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:51:46.895310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7db731-7df1-4cde-b159-4c319c46d5c6 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:51:46.954434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869ed38-c13b-418d-afec-455f171484fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:51:46.955375 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869ed38-c13b-418d-afec-455f171484fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:51:46.955661 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869ed38-c13b-418d-afec-455f171484fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:51:46.955943 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869ed38-c13b-418d-afec-455f171484fe admin admin] PATH_INFO: `/v3/users/eddfaa0c35784e91b39251b4e7c1751c` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:51:46.972105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869ed38-c13b-418d-afec-455f171484fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=eddfaa0c35784e91b39251b4e7c1751c)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:51:46.974140 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869ed38-c13b-418d-afec-455f171484fe admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:51:47.150918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-1869ed38-c13b-418d-afec-455f171484fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eddfaa0c35784e91b39251b4e7c1751c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:47.203110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-1869ed38-c13b-418d-afec-455f171484fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eddfaa0c35784e91b39251b4e7c1751c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:47.203745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-1869ed38-c13b-418d-afec-455f171484fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eddfaa0c35784e91b39251b4e7c1751c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:47.204499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-1869ed38-c13b-418d-afec-455f171484fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eddfaa0c35784e91b39251b4e7c1751c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:47.219206 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-1869ed38-c13b-418d-afec-455f171484fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eddfaa0c35784e91b39251b4e7c1751c'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:47.233834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1157/2335] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:51:46 2022] DELETE /identity/v3/users/eddfaa0c35784e91b39251b4e7c1751c => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:51:47.654503 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7448b0b-71e2-41ce-b9c8-21873481cc16 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:51:47.656315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-a7448b0b-71e2-41ce-b9c8-21873481cc16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:51:47.671022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7448b0b-71e2-41ce-b9c8-21873481cc16 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:51:47.781685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d13882-2c5b-41db-9a85-9649835faff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:51:47.793546 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d13882-2c5b-41db-9a85-9649835faff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:51:47.793546 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d13882-2c5b-41db-9a85-9649835faff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:51:47.793546 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d13882-2c5b-41db-9a85-9649835faff3 admin admin] PATH_INFO: `/v3/projects/eb0111e05bcb4e029aa001ef1aec20b4` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:51:47.801800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:51:47.809487 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d13882-2c5b-41db-9a85-9649835faff3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb0111e05bcb4e029aa001ef1aec20b4)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:51:47.812291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d13882-2c5b-41db-9a85-9649835faff3 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:51:47.947027 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-00d13882-2c5b-41db-9a85-9649835faff3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb0111e05bcb4e029aa001ef1aec20b4', 'request_id': 'req-00d13882-2c5b-41db-9a85-9649835faff3'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:47.957308 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-00d13882-2c5b-41db-9a85-9649835faff3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb0111e05bcb4e029aa001ef1aec20b4', 'request_id': 'req-00d13882-2c5b-41db-9a85-9649835faff3'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:47.964013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-00d13882-2c5b-41db-9a85-9649835faff3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb0111e05bcb4e029aa001ef1aec20b4', 'request_id': 'req-00d13882-2c5b-41db-9a85-9649835faff3'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:48.065783 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-00d13882-2c5b-41db-9a85-9649835faff3 admin admin] The token cache is being invalidate because project eb0111e05bcb4e029aa001ef1aec20b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:51:48.066159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-00d13882-2c5b-41db-9a85-9649835faff3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:48.066943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-00d13882-2c5b-41db-9a85-9649835faff3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:48.084264 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1179/2336] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:51:47 2022] DELETE /identity/v3/projects/eb0111e05bcb4e029aa001ef1aec20b4 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:51:48.319339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869ed38-c13b-418d-afec-455f171484fe admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:51:48.320821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1869ed38-c13b-418d-afec-455f171484fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:51:48.335008 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869ed38-c13b-418d-afec-455f171484fe admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:51:48.385196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:51:48.391860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f629ae-b3de-4370-8e25-688f51e30ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:51:48.399228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f629ae-b3de-4370-8e25-688f51e30ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:51:48.399228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f629ae-b3de-4370-8e25-688f51e30ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:51:48.399228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f629ae-b3de-4370-8e25-688f51e30ecb admin admin] PATH_INFO: `/v3/users/0ac3d9b2893b4f078c2a379696aaff34` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:51:48.399228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f629ae-b3de-4370-8e25-688f51e30ecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ac3d9b2893b4f078c2a379696aaff34)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:51:48.399228 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f629ae-b3de-4370-8e25-688f51e30ecb admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:51:48.651797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-05f629ae-b3de-4370-8e25-688f51e30ecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ac3d9b2893b4f078c2a379696aaff34'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:48.717200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-05f629ae-b3de-4370-8e25-688f51e30ecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ac3d9b2893b4f078c2a379696aaff34'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:48.717839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-05f629ae-b3de-4370-8e25-688f51e30ecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ac3d9b2893b4f078c2a379696aaff34'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:48.718449 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-05f629ae-b3de-4370-8e25-688f51e30ecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ac3d9b2893b4f078c2a379696aaff34'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:48.731326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-05f629ae-b3de-4370-8e25-688f51e30ecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ac3d9b2893b4f078c2a379696aaff34'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:48.744469 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1158/2337] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:51:48 2022] DELETE /identity/v3/users/0ac3d9b2893b4f078c2a379696aaff34 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:51:49.081931 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d13882-2c5b-41db-9a85-9649835faff3 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:51:49.083531 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-00d13882-2c5b-41db-9a85-9649835faff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:51:49.123736 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d13882-2c5b-41db-9a85-9649835faff3 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:51:49.194770 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f42375-2edb-409a-a67a-bcf8d7469d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:51:49.196851 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f42375-2edb-409a-a67a-bcf8d7469d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:51:49.197338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f42375-2edb-409a-a67a-bcf8d7469d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:51:49.197628 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f42375-2edb-409a-a67a-bcf8d7469d2a admin admin] PATH_INFO: `/v3/projects/dedc3b6e0d964cb9937f0a57894554b7` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:51:49.199744 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f42375-2edb-409a-a67a-bcf8d7469d2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dedc3b6e0d964cb9937f0a57894554b7)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:51:49.203511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f42375-2edb-409a-a67a-bcf8d7469d2a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:51:49.340555 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-73f42375-2edb-409a-a67a-bcf8d7469d2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dedc3b6e0d964cb9937f0a57894554b7', 'request_id': 'req-73f42375-2edb-409a-a67a-bcf8d7469d2a'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:49.346867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-73f42375-2edb-409a-a67a-bcf8d7469d2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dedc3b6e0d964cb9937f0a57894554b7', 'request_id': 'req-73f42375-2edb-409a-a67a-bcf8d7469d2a'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:49.356273 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-73f42375-2edb-409a-a67a-bcf8d7469d2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dedc3b6e0d964cb9937f0a57894554b7', 'request_id': 'req-73f42375-2edb-409a-a67a-bcf8d7469d2a'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:49.442865 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-73f42375-2edb-409a-a67a-bcf8d7469d2a admin admin] The token cache is being invalidate because project dedc3b6e0d964cb9937f0a57894554b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:51:49.444128 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-73f42375-2edb-409a-a67a-bcf8d7469d2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:49.444660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-73f42375-2edb-409a-a67a-bcf8d7469d2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:49.455762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1180/2338] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:51:49 2022] DELETE /identity/v3/projects/dedc3b6e0d964cb9937f0a57894554b7 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:51:49.455939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f629ae-b3de-4370-8e25-688f51e30ecb admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:51:49.455939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-05f629ae-b3de-4370-8e25-688f51e30ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:51:49.467203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f629ae-b3de-4370-8e25-688f51e30ecb admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:51:49.524071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9aa9b-d74d-4310-8f6c-e1ab520a3689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:51:49.525486 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9aa9b-d74d-4310-8f6c-e1ab520a3689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:51:49.525961 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9aa9b-d74d-4310-8f6c-e1ab520a3689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:51:49.526405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9aa9b-d74d-4310-8f6c-e1ab520a3689 admin admin] PATH_INFO: `/v3/users/d35becd699f947afa0b5305e3cb7f0c2` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:51:49.543861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9aa9b-d74d-4310-8f6c-e1ab520a3689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d35becd699f947afa0b5305e3cb7f0c2)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:51:49.545848 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab9aa9b-d74d-4310-8f6c-e1ab520a3689 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:51:49.679281 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6ab9aa9b-d74d-4310-8f6c-e1ab520a3689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd35becd699f947afa0b5305e3cb7f0c2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:49.723201 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6ab9aa9b-d74d-4310-8f6c-e1ab520a3689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd35becd699f947afa0b5305e3cb7f0c2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:49.723779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6ab9aa9b-d74d-4310-8f6c-e1ab520a3689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd35becd699f947afa0b5305e3cb7f0c2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:49.724433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6ab9aa9b-d74d-4310-8f6c-e1ab520a3689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd35becd699f947afa0b5305e3cb7f0c2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:49.733016 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:51:49.737567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6ab9aa9b-d74d-4310-8f6c-e1ab520a3689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd35becd699f947afa0b5305e3cb7f0c2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:49.749985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1159/2339] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 10:51:49 2022] DELETE /identity/v3/users/d35becd699f947afa0b5305e3cb7f0c2 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 10:51:49.993660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f42375-2edb-409a-a67a-bcf8d7469d2a admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:51:49.995161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-73f42375-2edb-409a-a67a-bcf8d7469d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:51:50.019770 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f42375-2edb-409a-a67a-bcf8d7469d2a admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:51:50.034459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:51:50.084903 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db3624-e8be-4841-a8ff-a1221f0febba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:51:50.086552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db3624-e8be-4841-a8ff-a1221f0febba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:51:50.086836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db3624-e8be-4841-a8ff-a1221f0febba admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:51:50.087112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db3624-e8be-4841-a8ff-a1221f0febba admin admin] PATH_INFO: `/v3/projects/dc3d8fedfb0c4d7c8742e5a1587ccadc` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:51:50.099216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db3624-e8be-4841-a8ff-a1221f0febba admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc3d8fedfb0c4d7c8742e5a1587ccadc)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:51:50.101180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db3624-e8be-4841-a8ff-a1221f0febba admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:51:50.234634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-20db3624-e8be-4841-a8ff-a1221f0febba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc3d8fedfb0c4d7c8742e5a1587ccadc', 'request_id': 'req-20db3624-e8be-4841-a8ff-a1221f0febba'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:50.242250 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-20db3624-e8be-4841-a8ff-a1221f0febba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc3d8fedfb0c4d7c8742e5a1587ccadc', 'request_id': 'req-20db3624-e8be-4841-a8ff-a1221f0febba'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:50.247960 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-20db3624-e8be-4841-a8ff-a1221f0febba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc3d8fedfb0c4d7c8742e5a1587ccadc', 'request_id': 'req-20db3624-e8be-4841-a8ff-a1221f0febba'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:50.328212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-20db3624-e8be-4841-a8ff-a1221f0febba admin admin] The token cache is being invalidate because project dc3d8fedfb0c4d7c8742e5a1587ccadc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 10:51:50.328499 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-20db3624-e8be-4841-a8ff-a1221f0febba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:50.329191 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-20db3624-e8be-4841-a8ff-a1221f0febba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 10:51:50.330787 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1181/2340] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 10:51:49 2022] DELETE /identity/v3/projects/dc3d8fedfb0c4d7c8742e5a1587ccadc => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 10:51:59.824687 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=7,keystone:DELETE=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:52:00.403572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:52:37.148800 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9aa9b-d74d-4310-8f6c-e1ab520a3689 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:52:37.150270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-6ab9aa9b-d74d-4310-8f6c-e1ab520a3689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:52:37.239525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab9aa9b-d74d-4310-8f6c-e1ab520a3689 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:52:37.308985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5147a8-c814-4036-bac7-bf0a4a450e72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:52:37.309712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5147a8-c814-4036-bac7-bf0a4a450e72 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:52:37.310000 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5147a8-c814-4036-bac7-bf0a4a450e72 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:52:37.310282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5147a8-c814-4036-bac7-bf0a4a450e72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:52:37.312390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-bf5147a8-c814-4036-bac7-bf0a4a450e72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:52:37.387124 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5147a8-c814-4036-bac7-bf0a4a450e72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:52:37.388708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5147a8-c814-4036-bac7-bf0a4a450e72 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:52:37.399424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1160/2341] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:52:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:52:47.394633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=94 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:53:38.021844 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db3624-e8be-4841-a8ff-a1221f0febba admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:53:38.023469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-20db3624-e8be-4841-a8ff-a1221f0febba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:53:38.094055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db3624-e8be-4841-a8ff-a1221f0febba admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:53:38.098335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3db6fd-8aa1-47b7-87cc-65be0b568851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:53:38.098921 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3db6fd-8aa1-47b7-87cc-65be0b568851 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:53:38.099206 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3db6fd-8aa1-47b7-87cc-65be0b568851 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:53:38.099484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3db6fd-8aa1-47b7-87cc-65be0b568851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:53:38.101482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-de3db6fd-8aa1-47b7-87cc-65be0b568851 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:53:38.116749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application [None req-de3db6fd-8aa1-47b7-87cc-65be0b568851 service cinder] Could not find project: 492be26275404bc3a439ae049bf4b9f4.: keystone.exception.ProjectNotFound: Could not find project: 492be26275404bc3a439ae049bf4b9f4. Jun 15 10:53:38.116749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 15 10:53:38.116749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 15 10:53:38.116749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 15 10:53:38.116749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 15 10:53:38.116749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 15 10:53:38.116749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 15 10:53:38.116749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 15 10:53:38.116749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 15 10:53:38.116749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 15 10:53:38.116749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 15 10:53:38.116749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 15 10:53:38.116749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jun 15 10:53:38.116749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jun 15 10:53:38.116749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:53:38.118303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self._validate_token_resources() Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 15 10:53:38.119920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.get_or_create( Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application with Lock( Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._enter() Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.creator() Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application created_value = creator( Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 15 10:53:38.121498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 15 10:53:38.122863 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 492be26275404bc3a439ae049bf4b9f4. Jun 15 10:53:38.122863 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: ERROR keystone.server.flask.application Jun 15 10:53:38.122863 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1182/2342] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:53:38 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 97 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jun 15 10:53:45.691567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5147a8-c814-4036-bac7-bf0a4a450e72 service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:53:45.693949 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-bf5147a8-c814-4036-bac7-bf0a4a450e72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:53:45.773183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5147a8-c814-4036-bac7-bf0a4a450e72 service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:53:45.776905 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3001edc-4d16-4faa-9d50-44f108562154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:53:45.777618 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3001edc-4d16-4faa-9d50-44f108562154 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:53:45.777911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3001edc-4d16-4faa-9d50-44f108562154 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:53:45.778192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3001edc-4d16-4faa-9d50-44f108562154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:53:45.780015 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c3001edc-4d16-4faa-9d50-44f108562154 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:53:45.795734 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3001edc-4d16-4faa-9d50-44f108562154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:53:45.796939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3001edc-4d16-4faa-9d50-44f108562154 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:53:45.809377 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1161/2343] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:53:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:53:46.504148 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3db6fd-8aa1-47b7-87cc-65be0b568851 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:53:46.504148 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-de3db6fd-8aa1-47b7-87cc-65be0b568851 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:53:46.515454 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3db6fd-8aa1-47b7-87cc-65be0b568851 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:53:46.519040 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad632837-0809-4bd8-8f48-1be426bea8d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:53:46.519677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad632837-0809-4bd8-8f48-1be426bea8d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:53:46.519958 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad632837-0809-4bd8-8f48-1be426bea8d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:53:46.520856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad632837-0809-4bd8-8f48-1be426bea8d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:53:46.522943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ad632837-0809-4bd8-8f48-1be426bea8d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:53:46.540095 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad632837-0809-4bd8-8f48-1be426bea8d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:53:46.542212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad632837-0809-4bd8-8f48-1be426bea8d3 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:53:46.554255 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1183/2344] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:53:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:53:51.061569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3001edc-4d16-4faa-9d50-44f108562154 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:53:51.063102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c3001edc-4d16-4faa-9d50-44f108562154 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:53:51.077324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3001edc-4d16-4faa-9d50-44f108562154 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:53:51.082966 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19256518-bc24-4838-9eb3-f650d2a92b58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:53:51.083623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19256518-bc24-4838-9eb3-f650d2a92b58 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:53:51.083914 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19256518-bc24-4838-9eb3-f650d2a92b58 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:53:51.084190 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19256518-bc24-4838-9eb3-f650d2a92b58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:53:51.086674 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-19256518-bc24-4838-9eb3-f650d2a92b58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:53:51.099824 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19256518-bc24-4838-9eb3-f650d2a92b58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:53:51.102777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19256518-bc24-4838-9eb3-f650d2a92b58 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:53:51.116748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1162/2345] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:53:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:53:56.548603 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=35 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:54:01.109323 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=37 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:54:12.529627 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad632837-0809-4bd8-8f48-1be426bea8d3 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:54:12.538850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad632837-0809-4bd8-8f48-1be426bea8d3 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:54:12.542806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcd009-edaf-4eef-8a39-ccd20ee49f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:54:12.543200 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcd009-edaf-4eef-8a39-ccd20ee49f66 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:54:12.543479 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcd009-edaf-4eef-8a39-ccd20ee49f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:54:12.543750 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcd009-edaf-4eef-8a39-ccd20ee49f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:54:12.547507 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5bdcd009-edaf-4eef-8a39-ccd20ee49f66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:54:12.558546 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcd009-edaf-4eef-8a39-ccd20ee49f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:54:12.559984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcd009-edaf-4eef-8a39-ccd20ee49f66 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:54:12.568710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1184/2346] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:54:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:54:22.564799 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=6 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:54:36.900381 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19256518-bc24-4838-9eb3-f650d2a92b58 service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:54:36.912234 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19256518-bc24-4838-9eb3-f650d2a92b58 service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:54:36.917832 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69a8b9-7fcd-4a6d-b15d-e7e0cd26670e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:54:36.918473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69a8b9-7fcd-4a6d-b15d-e7e0cd26670e service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:54:36.919330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69a8b9-7fcd-4a6d-b15d-e7e0cd26670e service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:54:36.919781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69a8b9-7fcd-4a6d-b15d-e7e0cd26670e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:54:36.921629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-df69a8b9-7fcd-4a6d-b15d-e7e0cd26670e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:54:36.994470 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69a8b9-7fcd-4a6d-b15d-e7e0cd26670e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:54:36.996222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69a8b9-7fcd-4a6d-b15d-e7e0cd26670e service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:54:37.005931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1163/2347] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:54:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:54:37.032525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcd009-edaf-4eef-8a39-ccd20ee49f66 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:54:37.034111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5bdcd009-edaf-4eef-8a39-ccd20ee49f66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:54:37.050787 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcd009-edaf-4eef-8a39-ccd20ee49f66 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:54:37.057079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336976fe-5411-40b8-9a24-b1e0fcc09df0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:54:37.057726 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336976fe-5411-40b8-9a24-b1e0fcc09df0 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:54:37.058015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336976fe-5411-40b8-9a24-b1e0fcc09df0 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:54:37.058278 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336976fe-5411-40b8-9a24-b1e0fcc09df0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:54:37.060300 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-336976fe-5411-40b8-9a24-b1e0fcc09df0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:54:37.145759 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336976fe-5411-40b8-9a24-b1e0fcc09df0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:54:37.147373 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336976fe-5411-40b8-9a24-b1e0fcc09df0 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:54:37.167367 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1185/2348] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:54:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:54:47.001350 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=33 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:54:47.164637 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=41 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:55:48.914541 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69a8b9-7fcd-4a6d-b15d-e7e0cd26670e service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:55:48.934891 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69a8b9-7fcd-4a6d-b15d-e7e0cd26670e service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:55:48.939468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b695f40-7eb5-4fa2-aaff-00dc5f30de8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:55:48.940463 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b695f40-7eb5-4fa2-aaff-00dc5f30de8b service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:55:48.940746 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b695f40-7eb5-4fa2-aaff-00dc5f30de8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:55:48.941050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b695f40-7eb5-4fa2-aaff-00dc5f30de8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:55:48.942939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1b695f40-7eb5-4fa2-aaff-00dc5f30de8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:55:49.032816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b695f40-7eb5-4fa2-aaff-00dc5f30de8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:55:49.034161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b695f40-7eb5-4fa2-aaff-00dc5f30de8b service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:55:49.055837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1164/2349] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:55:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:55:49.114520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336976fe-5411-40b8-9a24-b1e0fcc09df0 service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:55:49.150362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336976fe-5411-40b8-9a24-b1e0fcc09df0 service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:55:49.155266 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd3d23-77e9-45fe-b86d-f802cd9f8b82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:55:49.155801 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd3d23-77e9-45fe-b86d-f802cd9f8b82 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:55:49.155935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd3d23-77e9-45fe-b86d-f802cd9f8b82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:55:49.156194 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd3d23-77e9-45fe-b86d-f802cd9f8b82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:55:49.158339 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-19cd3d23-77e9-45fe-b86d-f802cd9f8b82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:55:49.238749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd3d23-77e9-45fe-b86d-f802cd9f8b82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:55:49.240209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cd3d23-77e9-45fe-b86d-f802cd9f8b82 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:55:49.261540 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1186/2350] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:55:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:55:59.051997 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=47 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:55:59.257158 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=41 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:56:24.813278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b695f40-7eb5-4fa2-aaff-00dc5f30de8b service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:56:24.814738 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1b695f40-7eb5-4fa2-aaff-00dc5f30de8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:56:24.926517 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b695f40-7eb5-4fa2-aaff-00dc5f30de8b service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:56:24.931911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72bca0-92f9-43db-bdde-a8c2ff94b31b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:56:24.932574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72bca0-92f9-43db-bdde-a8c2ff94b31b service placement] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:56:24.932854 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72bca0-92f9-43db-bdde-a8c2ff94b31b service placement] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:56:24.933158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72bca0-92f9-43db-bdde-a8c2ff94b31b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:56:24.934989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-dd72bca0-92f9-43db-bdde-a8c2ff94b31b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:56:24.949572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72bca0-92f9-43db-bdde-a8c2ff94b31b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:56:24.949572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72bca0-92f9-43db-bdde-a8c2ff94b31b service placement] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:56:24.960856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1165/2351] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:56:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:56:34.955842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=43 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:57:37.094203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd3d23-77e9-45fe-b86d-f802cd9f8b82 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:57:37.108190 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd3d23-77e9-45fe-b86d-f802cd9f8b82 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:57:37.114066 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7278c46-089f-4480-9a54-15c4ddbda850 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:57:37.114712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7278c46-089f-4480-9a54-15c4ddbda850 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:57:37.114998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7278c46-089f-4480-9a54-15c4ddbda850 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:57:37.115278 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7278c46-089f-4480-9a54-15c4ddbda850 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:57:37.124793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7278c46-089f-4480-9a54-15c4ddbda850 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:57:37.126427 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7278c46-089f-4480-9a54-15c4ddbda850 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:57:37.154355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1187/2352] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:57:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:57:40.741512 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72bca0-92f9-43db-bdde-a8c2ff94b31b service placement] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:57:40.760204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72bca0-92f9-43db-bdde-a8c2ff94b31b service placement] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:57:40.767388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6daf8-530e-4ffa-bf69-65f7c4ebac68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:57:40.768818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6daf8-530e-4ffa-bf69-65f7c4ebac68 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:57:40.769327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6daf8-530e-4ffa-bf69-65f7c4ebac68 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:57:40.769782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6daf8-530e-4ffa-bf69-65f7c4ebac68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:57:40.771938 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b2a6daf8-530e-4ffa-bf69-65f7c4ebac68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:57:40.787808 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6daf8-530e-4ffa-bf69-65f7c4ebac68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:57:40.790047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6daf8-530e-4ffa-bf69-65f7c4ebac68 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:57:40.803852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1166/2353] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:57:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:57:42.922141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333969b-b9cd-4d53-b7c9-0c852f083eaf None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:57:42.922676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333969b-b9cd-4d53-b7c9-0c852f083eaf None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:57:42.922830 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333969b-b9cd-4d53-b7c9-0c852f083eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:57:42.951639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-f333969b-b9cd-4d53-b7c9-0c852f083eaf None None] MFA Rules not processed for user `bbb69faf3ec54489a92fe87935f2f777`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:57:42.953333 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f333969b-b9cd-4d53-b7c9-0c852f083eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:57:42.960678 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1188/2354] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:57:43.287804 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6daf8-530e-4ffa-bf69-65f7c4ebac68 service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:57:43.308565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6daf8-530e-4ffa-bf69-65f7c4ebac68 service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:57:43.312553 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79542011-b8b4-44c4-98c3-be86fcd5905f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:57:43.313260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79542011-b8b4-44c4-98c3-be86fcd5905f service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:57:43.322668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79542011-b8b4-44c4-98c3-be86fcd5905f service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:57:43.322668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79542011-b8b4-44c4-98c3-be86fcd5905f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:57:43.322668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-79542011-b8b4-44c4-98c3-be86fcd5905f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:57:43.326578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79542011-b8b4-44c4-98c3-be86fcd5905f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:57:43.328055 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79542011-b8b4-44c4-98c3-be86fcd5905f service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:57:43.337281 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1167/2355] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:57:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:57:43.680837 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280f5e5-9032-4779-87d6-8427193196be None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:57:43.681181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280f5e5-9032-4779-87d6-8427193196be None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:57:43.681543 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0280f5e5-9032-4779-87d6-8427193196be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:57:43.711852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-0280f5e5-9032-4779-87d6-8427193196be None None] MFA Rules not processed for user `4beff9397579496c80fd6e93b7a81542`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:57:43.713917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0280f5e5-9032-4779-87d6-8427193196be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:57:43.721329 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1189/2356] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:57:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 10:57:43.792047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79542011-b8b4-44c4-98c3-be86fcd5905f service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:57:43.808116 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79542011-b8b4-44c4-98c3-be86fcd5905f service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:57:43.813372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed0cf2-cf13-4a6c-b589-92f7ed4961a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:57:43.817397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed0cf2-cf13-4a6c-b589-92f7ed4961a2 service placement] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:57:43.817864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed0cf2-cf13-4a6c-b589-92f7ed4961a2 service placement] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:57:43.818392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed0cf2-cf13-4a6c-b589-92f7ed4961a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:57:43.820636 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-51ed0cf2-cf13-4a6c-b589-92f7ed4961a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:57:43.843182 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed0cf2-cf13-4a6c-b589-92f7ed4961a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:57:43.844868 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed0cf2-cf13-4a6c-b589-92f7ed4961a2 service placement] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:57:43.855828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1168/2357] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:57:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:57:44.071293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280f5e5-9032-4779-87d6-8427193196be None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:57:44.081880 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0280f5e5-9032-4779-87d6-8427193196be None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:57:44.085499 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713f023-29b8-4e42-ae09-0c60969557b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:57:44.086167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713f023-29b8-4e42-ae09-0c60969557b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:57:44.086455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713f023-29b8-4e42-ae09-0c60969557b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:57:44.086735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713f023-29b8-4e42-ae09-0c60969557b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:57:44.089473 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-8713f023-29b8-4e42-ae09-0c60969557b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:57:44.100454 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713f023-29b8-4e42-ae09-0c60969557b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:57:44.101956 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713f023-29b8-4e42-ae09-0c60969557b2 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:57:44.110767 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1190/2358] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:57:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:57:53.851307 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=22 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:57:54.106775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=31 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:57:56.863151 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed0cf2-cf13-4a6c-b589-92f7ed4961a2 service placement] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:57:56.877982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed0cf2-cf13-4a6c-b589-92f7ed4961a2 service placement] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:57:56.883503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e2ddf-444a-444e-93cb-060f2b0cbcab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:57:56.884080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e2ddf-444a-444e-93cb-060f2b0cbcab service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:57:56.884433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e2ddf-444a-444e-93cb-060f2b0cbcab service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:57:56.884637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e2ddf-444a-444e-93cb-060f2b0cbcab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:57:56.887094 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-691e2ddf-444a-444e-93cb-060f2b0cbcab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:57:56.899464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e2ddf-444a-444e-93cb-060f2b0cbcab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:57:56.901189 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e2ddf-444a-444e-93cb-060f2b0cbcab service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:57:56.912134 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1169/2359] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:57:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:58:06.907638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=6 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:58:45.831614 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713f023-29b8-4e42-ae09-0c60969557b2 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:58:45.840620 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713f023-29b8-4e42-ae09-0c60969557b2 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:58:45.844912 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf78c0-96c2-4bab-a4cc-75939cbdea34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:58:45.845578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf78c0-96c2-4bab-a4cc-75939cbdea34 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:58:45.845858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf78c0-96c2-4bab-a4cc-75939cbdea34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:58:45.846133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf78c0-96c2-4bab-a4cc-75939cbdea34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:58:45.864811 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf78c0-96c2-4bab-a4cc-75939cbdea34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:58:45.866300 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf78c0-96c2-4bab-a4cc-75939cbdea34 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:58:45.893435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1191/2360] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:58:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 10:58:47.360704 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e2ddf-444a-444e-93cb-060f2b0cbcab service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:58:47.370387 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e2ddf-444a-444e-93cb-060f2b0cbcab service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:58:47.374203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f27ac-b517-4b87-a07b-23fc428b5cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:58:47.374950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f27ac-b517-4b87-a07b-23fc428b5cda service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:58:47.375237 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f27ac-b517-4b87-a07b-23fc428b5cda service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:58:47.375514 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f27ac-b517-4b87-a07b-23fc428b5cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:58:47.386752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f27ac-b517-4b87-a07b-23fc428b5cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:58:47.392053 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545f27ac-b517-4b87-a07b-23fc428b5cda service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:58:47.399850 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1170/2361] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:58:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:58:55.890518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=13 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:58:57.394739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=6 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:59:12.531560 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf78c0-96c2-4bab-a4cc-75939cbdea34 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:59:12.540945 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf78c0-96c2-4bab-a4cc-75939cbdea34 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:59:12.544819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abe87a-a11a-4011-b2d8-3f3aea14f142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:59:12.545514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abe87a-a11a-4011-b2d8-3f3aea14f142 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:59:12.545800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abe87a-a11a-4011-b2d8-3f3aea14f142 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:59:12.546081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abe87a-a11a-4011-b2d8-3f3aea14f142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:59:12.554899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30abe87a-a11a-4011-b2d8-3f3aea14f142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:59:12.556344 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30abe87a-a11a-4011-b2d8-3f3aea14f142 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:59:12.566278 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1192/2362] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:59:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:59:22.561446 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=6 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:59:31.245086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f27ac-b517-4b87-a07b-23fc428b5cda service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:59:31.256877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f27ac-b517-4b87-a07b-23fc428b5cda service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:59:31.263047 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14756f6-5e88-4bae-b527-36dd5cf8f29e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:59:31.263775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14756f6-5e88-4bae-b527-36dd5cf8f29e service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:59:31.264063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14756f6-5e88-4bae-b527-36dd5cf8f29e service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:59:31.264323 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14756f6-5e88-4bae-b527-36dd5cf8f29e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:59:31.275509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14756f6-5e88-4bae-b527-36dd5cf8f29e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:59:31.277552 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14756f6-5e88-4bae-b527-36dd5cf8f29e service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:59:31.290851 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1171/2363] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:59:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:59:34.903363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e388375-8402-401a-a8bf-acfd4470eedd None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:59:34.903363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e388375-8402-401a-a8bf-acfd4470eedd None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:59:34.903363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e388375-8402-401a-a8bf-acfd4470eedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:59:34.942499 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-9e388375-8402-401a-a8bf-acfd4470eedd None None] MFA Rules not processed for user `9f1e192f3eab42ee9f9ed5d2789e0d1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:59:34.947457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9e388375-8402-401a-a8bf-acfd4470eedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:59:34.953317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1193/2364] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 10:59:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:59:36.822340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14756f6-5e88-4bae-b527-36dd5cf8f29e service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:59:36.826100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a14756f6-5e88-4bae-b527-36dd5cf8f29e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:59:36.852352 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14756f6-5e88-4bae-b527-36dd5cf8f29e service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:59:36.863992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081834e1-f59a-4919-b6ba-cb4ee517dc76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:59:36.864749 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081834e1-f59a-4919-b6ba-cb4ee517dc76 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:59:36.865068 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081834e1-f59a-4919-b6ba-cb4ee517dc76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:59:36.865382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081834e1-f59a-4919-b6ba-cb4ee517dc76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:59:36.875326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081834e1-f59a-4919-b6ba-cb4ee517dc76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:59:36.877584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081834e1-f59a-4919-b6ba-cb4ee517dc76 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:59:36.890988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1172/2365] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:59:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:59:37.276060 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e388375-8402-401a-a8bf-acfd4470eedd None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:59:37.293040 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e388375-8402-401a-a8bf-acfd4470eedd None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:59:37.300805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765693b-e684-4192-98db-e14ec52558c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:59:37.300805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765693b-e684-4192-98db-e14ec52558c3 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:59:37.300805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765693b-e684-4192-98db-e14ec52558c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:59:37.301327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765693b-e684-4192-98db-e14ec52558c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:59:37.317599 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765693b-e684-4192-98db-e14ec52558c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:59:37.319346 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765693b-e684-4192-98db-e14ec52558c3 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 10:59:37.332671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1194/2366] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 10:59:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 10:59:46.886148 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=16 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:59:47.325570 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=16 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 10:59:59.417730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18d24d-9684-4f68-afbd-56fb7c1dac11 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:59:59.418299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18d24d-9684-4f68-afbd-56fb7c1dac11 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:59:59.418299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18d24d-9684-4f68-afbd-56fb7c1dac11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:59:59.487055 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-ec18d24d-9684-4f68-afbd-56fb7c1dac11 None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 10:59:59.488557 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ec18d24d-9684-4f68-afbd-56fb7c1dac11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:59:59.613676 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1173/2367] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jun 15 10:59:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 10:59:59.867509 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765693b-e684-4192-98db-e14ec52558c3 service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 10:59:59.869485 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5765693b-e684-4192-98db-e14ec52558c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 10:59:59.974684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765693b-e684-4192-98db-e14ec52558c3 service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 10:59:59.982432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbff889-2a30-485c-873a-55ed8bb8551f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 10:59:59.983281 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbff889-2a30-485c-873a-55ed8bb8551f service swift] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 10:59:59.983703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbff889-2a30-485c-873a-55ed8bb8551f service swift] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 10:59:59.983986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbff889-2a30-485c-873a-55ed8bb8551f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 10:59:59.996069 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbff889-2a30-485c-873a-55ed8bb8551f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 10:59:59.997765 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbff889-2a30-485c-873a-55ed8bb8551f service swift] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:00.010265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1195/2368] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 10:59:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 11:00:04.583671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec18d24d-9684-4f68-afbd-56fb7c1dac11 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:04.593544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec18d24d-9684-4f68-afbd-56fb7c1dac11 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:04.598941 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b9fa5-3f02-470e-876f-9eabdff862b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:04.600924 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b9fa5-3f02-470e-876f-9eabdff862b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:04.600924 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b9fa5-3f02-470e-876f-9eabdff862b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:04.600924 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b9fa5-3f02-470e-876f-9eabdff862b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:04.602973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e48b9fa5-3f02-470e-876f-9eabdff862b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:04.697025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b9fa5-3f02-470e-876f-9eabdff862b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:04.702358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b9fa5-3f02-470e-876f-9eabdff862b9 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:04.730358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1174/2369] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:00:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:00:06.684882 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbff889-2a30-485c-873a-55ed8bb8551f service swift] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:06.702724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbff889-2a30-485c-873a-55ed8bb8551f service swift] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:06.707347 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb4004-ab19-4163-b8b5-d8d2010630c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:06.708357 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb4004-ab19-4163-b8b5-d8d2010630c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:06.709067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb4004-ab19-4163-b8b5-d8d2010630c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:06.709789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb4004-ab19-4163-b8b5-d8d2010630c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:06.712871 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e7fb4004-ab19-4163-b8b5-d8d2010630c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:06.729239 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb4004-ab19-4163-b8b5-d8d2010630c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:06.731584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb4004-ab19-4163-b8b5-d8d2010630c2 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:06.750096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1196/2370] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:00:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:00:07.649293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b9fa5-3f02-470e-876f-9eabdff862b9 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:07.673177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b9fa5-3f02-470e-876f-9eabdff862b9 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:07.689577 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0a142-542c-4712-97bb-4a1460cf4b27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:07.692243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0a142-542c-4712-97bb-4a1460cf4b27 service placement] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:07.693668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0a142-542c-4712-97bb-4a1460cf4b27 service placement] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:07.697225 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0a142-542c-4712-97bb-4a1460cf4b27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:07.703188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9cf0a142-542c-4712-97bb-4a1460cf4b27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:07.718855 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0a142-542c-4712-97bb-4a1460cf4b27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:07.720827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0a142-542c-4712-97bb-4a1460cf4b27 service placement] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:07.744156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1175/2371] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:00:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:00:07.991894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb4004-ab19-4163-b8b5-d8d2010630c2 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:07.993445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e7fb4004-ab19-4163-b8b5-d8d2010630c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:08.014091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb4004-ab19-4163-b8b5-d8d2010630c2 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:08.029119 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b126d46-c56d-41b6-94b3-6f53975ba3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:08.030432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b126d46-c56d-41b6-94b3-6f53975ba3b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:08.031019 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b126d46-c56d-41b6-94b3-6f53975ba3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:08.031554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b126d46-c56d-41b6-94b3-6f53975ba3b1 admin admin] PATH_INFO: `/v3/users/80a318aa610e449fb6e480372ee61db8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:08.033502 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b126d46-c56d-41b6-94b3-6f53975ba3b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80a318aa610e449fb6e480372ee61db8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:08.036327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b126d46-c56d-41b6-94b3-6f53975ba3b1 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:08.249714 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6b126d46-c56d-41b6-94b3-6f53975ba3b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80a318aa610e449fb6e480372ee61db8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:08.287807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6b126d46-c56d-41b6-94b3-6f53975ba3b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80a318aa610e449fb6e480372ee61db8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:08.398243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6b126d46-c56d-41b6-94b3-6f53975ba3b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80a318aa610e449fb6e480372ee61db8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:08.408406 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6b126d46-c56d-41b6-94b3-6f53975ba3b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80a318aa610e449fb6e480372ee61db8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:08.410817 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6b126d46-c56d-41b6-94b3-6f53975ba3b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80a318aa610e449fb6e480372ee61db8'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:08.445609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1197/2372] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:00:07 2022] DELETE /identity/v3/users/80a318aa610e449fb6e480372ee61db8 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:09.103921 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0a142-542c-4712-97bb-4a1460cf4b27 service placement] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:09.118701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0a142-542c-4712-97bb-4a1460cf4b27 service placement] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:09.198792 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:09.206435 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebc55f-2d06-442e-b331-6c6d9c9db886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:09.208428 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebc55f-2d06-442e-b331-6c6d9c9db886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:09.208811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebc55f-2d06-442e-b331-6c6d9c9db886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:09.209200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebc55f-2d06-442e-b331-6c6d9c9db886 admin admin] PATH_INFO: `/v3/projects/40cd185858444d8aaf78e72a740b624f` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:09.227564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebc55f-2d06-442e-b331-6c6d9c9db886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40cd185858444d8aaf78e72a740b624f)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:09.229926 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebc55f-2d06-442e-b331-6c6d9c9db886 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:09.414818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-62ebc55f-2d06-442e-b331-6c6d9c9db886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40cd185858444d8aaf78e72a740b624f', 'request_id': 'req-62ebc55f-2d06-442e-b331-6c6d9c9db886'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:09.423441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-62ebc55f-2d06-442e-b331-6c6d9c9db886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40cd185858444d8aaf78e72a740b624f', 'request_id': 'req-62ebc55f-2d06-442e-b331-6c6d9c9db886'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:09.523155 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-62ebc55f-2d06-442e-b331-6c6d9c9db886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40cd185858444d8aaf78e72a740b624f', 'request_id': 'req-62ebc55f-2d06-442e-b331-6c6d9c9db886'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:09.548142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-62ebc55f-2d06-442e-b331-6c6d9c9db886 admin admin] The token cache is being invalidate because project 40cd185858444d8aaf78e72a740b624f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:00:09.548868 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-62ebc55f-2d06-442e-b331-6c6d9c9db886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:09.551485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-62ebc55f-2d06-442e-b331-6c6d9c9db886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:09.556006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1176/2373] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:00:09 2022] DELETE /identity/v3/projects/40cd185858444d8aaf78e72a740b624f => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:10.768331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b126d46-c56d-41b6-94b3-6f53975ba3b1 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:10.770888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6b126d46-c56d-41b6-94b3-6f53975ba3b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:10.806470 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b126d46-c56d-41b6-94b3-6f53975ba3b1 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:10.867282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:10.948276 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb301ac-fff9-455b-9884-7a783fbc5c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:10.952981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb301ac-fff9-455b-9884-7a783fbc5c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:10.954122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb301ac-fff9-455b-9884-7a783fbc5c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:10.956033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb301ac-fff9-455b-9884-7a783fbc5c37 admin admin] PATH_INFO: `/v3/users/5090fa81be4d41bbb62da2c4f273d537` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:10.958361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb301ac-fff9-455b-9884-7a783fbc5c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5090fa81be4d41bbb62da2c4f273d537)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:10.961100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb301ac-fff9-455b-9884-7a783fbc5c37 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:11.168913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3cb301ac-fff9-455b-9884-7a783fbc5c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5090fa81be4d41bbb62da2c4f273d537'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:11.198244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3cb301ac-fff9-455b-9884-7a783fbc5c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5090fa81be4d41bbb62da2c4f273d537'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:11.280287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3cb301ac-fff9-455b-9884-7a783fbc5c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5090fa81be4d41bbb62da2c4f273d537'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:11.290260 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3cb301ac-fff9-455b-9884-7a783fbc5c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5090fa81be4d41bbb62da2c4f273d537'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:11.291067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3cb301ac-fff9-455b-9884-7a783fbc5c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5090fa81be4d41bbb62da2c4f273d537'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:11.309526 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1198/2374] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:00:10 2022] DELETE /identity/v3/users/5090fa81be4d41bbb62da2c4f273d537 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:11.876457 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebc55f-2d06-442e-b331-6c6d9c9db886 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:11.881415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-62ebc55f-2d06-442e-b331-6c6d9c9db886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:11.902824 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebc55f-2d06-442e-b331-6c6d9c9db886 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:12.009995 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969ec04-131e-474d-9d33-fe2abb10dfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:12.011267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969ec04-131e-474d-9d33-fe2abb10dfa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:12.011697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969ec04-131e-474d-9d33-fe2abb10dfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:12.013948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969ec04-131e-474d-9d33-fe2abb10dfa7 admin admin] PATH_INFO: `/v3/projects/ab32692641974828b6fa534495acca01` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:12.019861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969ec04-131e-474d-9d33-fe2abb10dfa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab32692641974828b6fa534495acca01)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:12.020559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969ec04-131e-474d-9d33-fe2abb10dfa7 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:12.190267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e969ec04-131e-474d-9d33-fe2abb10dfa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab32692641974828b6fa534495acca01', 'request_id': 'req-e969ec04-131e-474d-9d33-fe2abb10dfa7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:12.200671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e969ec04-131e-474d-9d33-fe2abb10dfa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab32692641974828b6fa534495acca01', 'request_id': 'req-e969ec04-131e-474d-9d33-fe2abb10dfa7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:12.302585 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e969ec04-131e-474d-9d33-fe2abb10dfa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab32692641974828b6fa534495acca01', 'request_id': 'req-e969ec04-131e-474d-9d33-fe2abb10dfa7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:12.305834 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1630c-6345-4c4b-a15d-8edd5d949115 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:12.306958 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1630c-6345-4c4b-a15d-8edd5d949115 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:12.307608 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1630c-6345-4c4b-a15d-8edd5d949115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:12.318411 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e969ec04-131e-474d-9d33-fe2abb10dfa7 admin admin] The token cache is being invalidate because project ab32692641974828b6fa534495acca01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:00:12.318944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e969ec04-131e-474d-9d33-fe2abb10dfa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:12.322537 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e969ec04-131e-474d-9d33-fe2abb10dfa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:12.327607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-49c1630c-6345-4c4b-a15d-8edd5d949115 None None] MFA Rules not processed for user `4beff9397579496c80fd6e93b7a81542`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:12.334759 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1177/2375] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:00:11 2022] DELETE /identity/v3/projects/ab32692641974828b6fa534495acca01 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:12.334958 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-49c1630c-6345-4c4b-a15d-8edd5d949115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:12.335064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839ddaa-acf0-4b5f-8b30-7ef0a3b6d730 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:12.335064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839ddaa-acf0-4b5f-8b30-7ef0a3b6d730 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:12.335064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839ddaa-acf0-4b5f-8b30-7ef0a3b6d730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:12.338924 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:12.395954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-4839ddaa-acf0-4b5f-8b30-7ef0a3b6d730 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:12.397494 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4839ddaa-acf0-4b5f-8b30-7ef0a3b6d730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:12.414617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1199/2376] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:00:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:12.496669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1178/2377] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:12.505505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb19b1d-8615-4418-a945-6d13598809a7 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:12.505802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb19b1d-8615-4418-a945-6d13598809a7 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:12.506087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb19b1d-8615-4418-a945-6d13598809a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:12.534045 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-6bb19b1d-8615-4418-a945-6d13598809a7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:12.536225 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6bb19b1d-8615-4418-a945-6d13598809a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:12.548560 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1200/2378] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:12.553938 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839ddaa-acf0-4b5f-8b30-7ef0a3b6d730 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:12.565039 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839ddaa-acf0-4b5f-8b30-7ef0a3b6d730 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:12.571367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfee2cd-3d77-4ea2-9600-caa30c7ba9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:12.572187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfee2cd-3d77-4ea2-9600-caa30c7ba9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:12.572729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfee2cd-3d77-4ea2-9600-caa30c7ba9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:12.573025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfee2cd-3d77-4ea2-9600-caa30c7ba9e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:12.573772 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfee2cd-3d77-4ea2-9600-caa30c7ba9e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:00:12.574208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfee2cd-3d77-4ea2-9600-caa30c7ba9e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:12.576051 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfee2cd-3d77-4ea2-9600-caa30c7ba9e9 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:12.594089 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1179/2379] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:00:12.598218 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb19b1d-8615-4418-a945-6d13598809a7 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:12.608933 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb19b1d-8615-4418-a945-6d13598809a7 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:12.618197 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7a10a-2ba7-400b-9f0b-fc4a32834708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:12.618893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7a10a-2ba7-400b-9f0b-fc4a32834708 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:12.619196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7a10a-2ba7-400b-9f0b-fc4a32834708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:12.619445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7a10a-2ba7-400b-9f0b-fc4a32834708 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:12.620147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f7a10a-2ba7-400b-9f0b-fc4a32834708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:12.622121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f7a10a-2ba7-400b-9f0b-fc4a32834708 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:12.895095 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1201/2380] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:00:12 2022] POST /identity/v3/projects => generated 381 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:00:12.896300 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfee2cd-3d77-4ea2-9600-caa30c7ba9e9 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:12.906950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfee2cd-3d77-4ea2-9600-caa30c7ba9e9 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:12.980885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401420e-2139-4d3c-a801-08af5dcead1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:12.981703 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401420e-2139-4d3c-a801-08af5dcead1f admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:12.982127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401420e-2139-4d3c-a801-08af5dcead1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:12.982683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401420e-2139-4d3c-a801-08af5dcead1f admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:12.983498 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401420e-2139-4d3c-a801-08af5dcead1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:12.991562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401420e-2139-4d3c-a801-08af5dcead1f admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:13.117194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1180/2381] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:00:12 2022] POST /identity/v3/users => generated 291 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:00:13.118275 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7a10a-2ba7-400b-9f0b-fc4a32834708 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:13.138014 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7a10a-2ba7-400b-9f0b-fc4a32834708 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:13.149652 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaadce1d-0abd-4bdd-beba-9e6839f0ad71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:13.150433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadce1d-0abd-4bdd-beba-9e6839f0ad71 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:13.150776 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadce1d-0abd-4bdd-beba-9e6839f0ad71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:13.151251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadce1d-0abd-4bdd-beba-9e6839f0ad71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:13.152065 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaadce1d-0abd-4bdd-beba-9e6839f0ad71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:13.153850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaadce1d-0abd-4bdd-beba-9e6839f0ad71 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:13.172428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1202/2382] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:00:13.180211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401420e-2139-4d3c-a801-08af5dcead1f admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:13.203714 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401420e-2139-4d3c-a801-08af5dcead1f admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:13.208888 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4479c6-5175-40e4-9915-cb702cbd862d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:13.209684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4479c6-5175-40e4-9915-cb702cbd862d admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:13.210053 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4479c6-5175-40e4-9915-cb702cbd862d admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:13.210238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4479c6-5175-40e4-9915-cb702cbd862d admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:13.210978 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4479c6-5175-40e4-9915-cb702cbd862d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:13.212635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4479c6-5175-40e4-9915-cb702cbd862d admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:13.227865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1181/2383] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:00:13.232168 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaadce1d-0abd-4bdd-beba-9e6839f0ad71 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:13.240258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:13.245737 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaadce1d-0abd-4bdd-beba-9e6839f0ad71 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:13.251192 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a63b1f-af0e-4f14-8469-52dfb7377086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:13.254670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a63b1f-af0e-4f14-8469-52dfb7377086 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:13.255153 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a63b1f-af0e-4f14-8469-52dfb7377086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:13.255561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a63b1f-af0e-4f14-8469-52dfb7377086 admin admin] PATH_INFO: `/v3/projects/70550ab7423c41f6a4d6a6f5d08e84b5/users/ea037445944841f78fce862081119626/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:13.291676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a63b1f-af0e-4f14-8469-52dfb7377086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70550ab7423c41f6a4d6a6f5d08e84b5, user_id=ea037445944841f78fce862081119626, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:13.295061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a63b1f-af0e-4f14-8469-52dfb7377086 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:13.405624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1203/2384] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:00:13 2022] PUT /identity/v3/projects/70550ab7423c41f6a4d6a6f5d08e84b5/users/ea037445944841f78fce862081119626/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:00:13.407395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206a8c-2e2d-426b-abce-d0452ae1bc3d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:13.409048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206a8c-2e2d-426b-abce-d0452ae1bc3d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:13.409407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206a8c-2e2d-426b-abce-d0452ae1bc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:13.453057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-30206a8c-2e2d-426b-abce-d0452ae1bc3d None None] MFA Rules not processed for user `ea037445944841f78fce862081119626`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:13.454355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-30206a8c-2e2d-426b-abce-d0452ae1bc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:13.514603 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:13.549250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1182/2385] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:00:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:13.554887 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a63b1f-af0e-4f14-8469-52dfb7377086 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:13.637180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a63b1f-af0e-4f14-8469-52dfb7377086 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:13.642615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdbed8-cdb9-4693-bb80-8e44b167915b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:13.643374 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdbed8-cdb9-4693-bb80-8e44b167915b admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:13.643735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdbed8-cdb9-4693-bb80-8e44b167915b admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:13.644680 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdbed8-cdb9-4693-bb80-8e44b167915b admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:13.645603 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdbed8-cdb9-4693-bb80-8e44b167915b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:13.647685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdbed8-cdb9-4693-bb80-8e44b167915b admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:13.836028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1204/2386] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:00:13 2022] POST /identity/v3/projects => generated 379 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:00:13.837315 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30206a8c-2e2d-426b-abce-d0452ae1bc3d None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:13.850934 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30206a8c-2e2d-426b-abce-d0452ae1bc3d None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:13.912664 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725d621-4a4e-4dee-9252-b8510fa2bf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:13.913404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725d621-4a4e-4dee-9252-b8510fa2bf5a admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:13.913695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725d621-4a4e-4dee-9252-b8510fa2bf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:13.913971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725d621-4a4e-4dee-9252-b8510fa2bf5a admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:13.914722 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725d621-4a4e-4dee-9252-b8510fa2bf5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:13.916827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725d621-4a4e-4dee-9252-b8510fa2bf5a admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:14.074371 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1183/2387] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:00:13 2022] POST /identity/v3/users => generated 296 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:00:14.078503 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdbed8-cdb9-4693-bb80-8e44b167915b admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:14.090160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdbed8-cdb9-4693-bb80-8e44b167915b admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:14.099417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c02cd7-364b-4fbd-9aa2-24b0dd2009d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:14.099417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c02cd7-364b-4fbd-9aa2-24b0dd2009d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:14.099417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c02cd7-364b-4fbd-9aa2-24b0dd2009d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:14.099417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c02cd7-364b-4fbd-9aa2-24b0dd2009d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:14.100017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c02cd7-364b-4fbd-9aa2-24b0dd2009d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:14.101516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c02cd7-364b-4fbd-9aa2-24b0dd2009d5 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:14.121698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1205/2388] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:00:14.123825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725d621-4a4e-4dee-9252-b8510fa2bf5a admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:14.136106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725d621-4a4e-4dee-9252-b8510fa2bf5a admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:14.140818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82275c3-8189-4897-b658-714cc92106b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:14.143023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82275c3-8189-4897-b658-714cc92106b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:14.143304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82275c3-8189-4897-b658-714cc92106b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:14.143583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82275c3-8189-4897-b658-714cc92106b4 admin admin] PATH_INFO: `/v3/projects/d8e7863764a44ef794f445cfa97acbec/users/91ac861547e14ed5b944c306b71c1fd0/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:14.161833 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82275c3-8189-4897-b658-714cc92106b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e7863764a44ef794f445cfa97acbec, user_id=91ac861547e14ed5b944c306b71c1fd0, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:14.164597 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82275c3-8189-4897-b658-714cc92106b4 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:14.273714 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1184/2389] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:00:14 2022] PUT /identity/v3/projects/d8e7863764a44ef794f445cfa97acbec/users/91ac861547e14ed5b944c306b71c1fd0/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:14.283424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c02cd7-364b-4fbd-9aa2-24b0dd2009d5 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:14.294727 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c02cd7-364b-4fbd-9aa2-24b0dd2009d5 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:14.373070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f4089-d6a6-4a4d-812c-0c000d915132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:14.373756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f4089-d6a6-4a4d-812c-0c000d915132 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:14.374065 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f4089-d6a6-4a4d-812c-0c000d915132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:14.374417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f4089-d6a6-4a4d-812c-0c000d915132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:14.375247 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f4089-d6a6-4a4d-812c-0c000d915132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:14.377033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f4089-d6a6-4a4d-812c-0c000d915132 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:14.388126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1206/2390] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:00:14.392291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82275c3-8189-4897-b658-714cc92106b4 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:14.404551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82275c3-8189-4897-b658-714cc92106b4 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:14.409691 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f944330-912a-43fa-bad4-a545a1dee28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:14.411245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f944330-912a-43fa-bad4-a545a1dee28c admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:14.413781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f944330-912a-43fa-bad4-a545a1dee28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:14.413781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f944330-912a-43fa-bad4-a545a1dee28c admin admin] PATH_INFO: `/v3/projects/d8e7863764a44ef794f445cfa97acbec/users/91ac861547e14ed5b944c306b71c1fd0/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:14.414188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f944330-912a-43fa-bad4-a545a1dee28c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8e7863764a44ef794f445cfa97acbec, user_id=91ac861547e14ed5b944c306b71c1fd0, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:14.416584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f944330-912a-43fa-bad4-a545a1dee28c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:14.428112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1185/2391] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:00:14 2022] PUT /identity/v3/projects/d8e7863764a44ef794f445cfa97acbec/users/91ac861547e14ed5b944c306b71c1fd0/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:14.440748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5854d-9252-4b3a-a16b-bfa9dfbeddd3 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:14.441384 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5854d-9252-4b3a-a16b-bfa9dfbeddd3 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:14.441384 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5854d-9252-4b3a-a16b-bfa9dfbeddd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:14.471387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-cbf5854d-9252-4b3a-a16b-bfa9dfbeddd3 None None] MFA Rules not processed for user `91ac861547e14ed5b944c306b71c1fd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:14.473434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-cbf5854d-9252-4b3a-a16b-bfa9dfbeddd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:14.530954 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:14.565342 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1207/2392] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:00:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:14.577334 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f944330-912a-43fa-bad4-a545a1dee28c admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:14.579052 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-8f944330-912a-43fa-bad4-a545a1dee28c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:14.645442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f944330-912a-43fa-bad4-a545a1dee28c admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:14.649199 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a43f08-018f-4d8e-afcb-d18118ec1f9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:14.649854 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a43f08-018f-4d8e-afcb-d18118ec1f9a service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:14.650141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a43f08-018f-4d8e-afcb-d18118ec1f9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:14.650414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a43f08-018f-4d8e-afcb-d18118ec1f9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:14.661121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a43f08-018f-4d8e-afcb-d18118ec1f9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:14.662568 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a43f08-018f-4d8e-afcb-d18118ec1f9a service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:14.672920 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1186/2393] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:00:14 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:00:14.904761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5854d-9252-4b3a-a16b-bfa9dfbeddd3 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:14.915622 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5854d-9252-4b3a-a16b-bfa9dfbeddd3 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:14.919763 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c45b69-03ff-4680-a4bd-26204e707bb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:14.920496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c45b69-03ff-4680-a4bd-26204e707bb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:14.920838 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c45b69-03ff-4680-a4bd-26204e707bb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:14.921214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c45b69-03ff-4680-a4bd-26204e707bb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:14.983225 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c45b69-03ff-4680-a4bd-26204e707bb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:14.984677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c45b69-03ff-4680-a4bd-26204e707bb9 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:14.997466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1208/2394] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:00:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:00:22.280681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914415d-68cf-40e7-8362-f40be69b7a65 None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:22.281485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914415d-68cf-40e7-8362-f40be69b7a65 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:22.281485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0914415d-68cf-40e7-8362-f40be69b7a65 None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:22.282678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1187/2395] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jun 15 11:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 11:00:22.289539 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e071b-90ea-4b5c-b97a-3dbaed371a6e None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:22.290105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e071b-90ea-4b5c-b97a-3dbaed371a6e None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:22.290227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e071b-90ea-4b5c-b97a-3dbaed371a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:22.329359 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-0e5e071b-90ea-4b5c-b97a-3dbaed371a6e None None] MFA Rules not processed for user `26f23f8595fc43ad94aee7d0192164a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:22.330628 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0e5e071b-90ea-4b5c-b97a-3dbaed371a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:22.399771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1209/2396] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jun 15 11:00:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:22.411585 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914415d-68cf-40e7-8362-f40be69b7a65 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:22.413339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-0914415d-68cf-40e7-8362-f40be69b7a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:22.452734 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:22.494954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0914415d-68cf-40e7-8362-f40be69b7a65 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:22.498492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0602c-c7f0-4d70-889d-f39b2ce5c41f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fbf33189aa94f4f973d558d615a23e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:22.499133 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0602c-c7f0-4d70-889d-f39b2ce5c41f service glance] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:22.499413 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0602c-c7f0-4d70-889d-f39b2ce5c41f service glance] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:22.499683 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0602c-c7f0-4d70-889d-f39b2ce5c41f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:22.510084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0602c-c7f0-4d70-889d-f39b2ce5c41f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:22.511520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0602c-c7f0-4d70-889d-f39b2ce5c41f service glance] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:22.522950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1188/2397] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:00:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:00:22.750336 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaed7ec-648f-44fd-b323-8ffaf9243267 None None] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:22.750606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaed7ec-648f-44fd-b323-8ffaf9243267 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:22.750800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaed7ec-648f-44fd-b323-8ffaf9243267 None None] PATH_INFO: `/` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:22.752307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1210/2398] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jun 15 11:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 11:00:22.757373 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6144ecf-10b7-4bfe-9c31-87aa857031a8 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:22.757750 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6144ecf-10b7-4bfe-9c31-87aa857031a8 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:22.758035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6144ecf-10b7-4bfe-9c31-87aa857031a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:22.795909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b6144ecf-10b7-4bfe-9c31-87aa857031a8 None None] MFA Rules not processed for user `ae926ce83d464c768baddd90de31aa40`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:22.797160 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b6144ecf-10b7-4bfe-9c31-87aa857031a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:22.852410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1189/2399] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jun 15 11:00:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:22.864829 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaed7ec-648f-44fd-b323-8ffaf9243267 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:22.866599 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-aeaed7ec-648f-44fd-b323-8ffaf9243267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:22.880833 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaed7ec-648f-44fd-b323-8ffaf9243267 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:22.885687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ad777-df15-4452-ac75-9d6e2282a6b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:22.885687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ad777-df15-4452-ac75-9d6e2282a6b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:22.885687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ad777-df15-4452-ac75-9d6e2282a6b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:22.886100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ad777-df15-4452-ac75-9d6e2282a6b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:22.893730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ad777-df15-4452-ac75-9d6e2282a6b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:22.896399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ad777-df15-4452-ac75-9d6e2282a6b0 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:22.904921 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1211/2400] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:00:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 11:00:22.991790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48577fb1-a4bc-4208-a057-e50c69390e2b None None] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:22.992158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48577fb1-a4bc-4208-a057-e50c69390e2b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:22.992450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48577fb1-a4bc-4208-a057-e50c69390e2b None None] PATH_INFO: `/` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:22.993909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1190/2401] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jun 15 11:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 15 11:00:23.000104 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51815eb-4a16-4020-8cf9-f6c4a51902ee None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:23.000532 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51815eb-4a16-4020-8cf9-f6c4a51902ee None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:23.000762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51815eb-4a16-4020-8cf9-f6c4a51902ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:23.021631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-f51815eb-4a16-4020-8cf9-f6c4a51902ee None None] MFA Rules not processed for user `26f23f8595fc43ad94aee7d0192164a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:23.022846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f51815eb-4a16-4020-8cf9-f6c4a51902ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:23.038919 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1212/2402] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jun 15 11:00:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:23.049525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48577fb1-a4bc-4208-a057-e50c69390e2b None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:23.057561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48577fb1-a4bc-4208-a057-e50c69390e2b None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:23.061254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6769b-3449-426b-aa04-9035ff815023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:23.061961 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6769b-3449-426b-aa04-9035ff815023 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:23.062172 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6769b-3449-426b-aa04-9035ff815023 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:23.062453 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6769b-3449-426b-aa04-9035ff815023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:23.074395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6769b-3449-426b-aa04-9035ff815023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:23.077788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6769b-3449-426b-aa04-9035ff815023 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:23.093229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1191/2403] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:00:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 11:00:25.913261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3ed5a-f21d-44fd-b2bd-12580dd555f5 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:25.913261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3ed5a-f21d-44fd-b2bd-12580dd555f5 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:25.913261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3ed5a-f21d-44fd-b2bd-12580dd555f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:25.932570 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-feb3ed5a-f21d-44fd-b2bd-12580dd555f5 None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:25.933814 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-feb3ed5a-f21d-44fd-b2bd-12580dd555f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:25.966982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:25.997242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1213/2404] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jun 15 11:00:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:26.007727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6769b-3449-426b-aa04-9035ff815023 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:26.009345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c4e6769b-3449-426b-aa04-9035ff815023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:26.089878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6769b-3449-426b-aa04-9035ff815023 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:26.095593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aea7b3-6a3f-42cc-8dc1-f6e76fe7cce3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:26.095786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aea7b3-6a3f-42cc-8dc1-f6e76fe7cce3 service swift] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:26.096025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aea7b3-6a3f-42cc-8dc1-f6e76fe7cce3 service swift] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:26.096357 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aea7b3-6a3f-42cc-8dc1-f6e76fe7cce3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:26.115369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aea7b3-6a3f-42cc-8dc1-f6e76fe7cce3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:26.117311 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aea7b3-6a3f-42cc-8dc1-f6e76fe7cce3 service swift] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:26.129554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1192/2405] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 11:00:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 11:00:29.181647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3ed5a-f21d-44fd-b2bd-12580dd555f5 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:29.267704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3ed5a-f21d-44fd-b2bd-12580dd555f5 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:29.273057 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51704db-6042-417e-91d2-55f0f20ded5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:29.273947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51704db-6042-417e-91d2-55f0f20ded5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:29.274210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51704db-6042-417e-91d2-55f0f20ded5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:29.274484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51704db-6042-417e-91d2-55f0f20ded5a admin admin] PATH_INFO: `/v3/users/ea037445944841f78fce862081119626` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:29.275691 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51704db-6042-417e-91d2-55f0f20ded5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea037445944841f78fce862081119626)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:29.277655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51704db-6042-417e-91d2-55f0f20ded5a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:29.476646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a51704db-6042-417e-91d2-55f0f20ded5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea037445944841f78fce862081119626'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:29.491364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a51704db-6042-417e-91d2-55f0f20ded5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea037445944841f78fce862081119626'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:29.570325 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a51704db-6042-417e-91d2-55f0f20ded5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea037445944841f78fce862081119626'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:29.582355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a51704db-6042-417e-91d2-55f0f20ded5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea037445944841f78fce862081119626'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:29.583523 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a51704db-6042-417e-91d2-55f0f20ded5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea037445944841f78fce862081119626'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:29.595017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1214/2406] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:00:29 2022] DELETE /identity/v3/users/ea037445944841f78fce862081119626 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:29.599710 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4ab9b-4e5b-4a57-8d15-3e419fef0fc2 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:29.600262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4ab9b-4e5b-4a57-8d15-3e419fef0fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:29.600394 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4ab9b-4e5b-4a57-8d15-3e419fef0fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:29.618607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-c0f4ab9b-4e5b-4a57-8d15-3e419fef0fc2 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:29.619840 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c0f4ab9b-4e5b-4a57-8d15-3e419fef0fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:29.668662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:29.724940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1193/2407] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:29.736912 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51704db-6042-417e-91d2-55f0f20ded5a admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:29.739347 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-a51704db-6042-417e-91d2-55f0f20ded5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:29.802013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51704db-6042-417e-91d2-55f0f20ded5a admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:29.805806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e7836-9165-4a5b-aee7-b5d79ba25a78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:29.806473 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e7836-9165-4a5b-aee7-b5d79ba25a78 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:29.806756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e7836-9165-4a5b-aee7-b5d79ba25a78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:29.807035 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e7836-9165-4a5b-aee7-b5d79ba25a78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:29.817078 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e7836-9165-4a5b-aee7-b5d79ba25a78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:29.818555 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092e7836-9165-4a5b-aee7-b5d79ba25a78 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:29.823643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:29.830353 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1215/2408] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:00:30.298764 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4ab9b-4e5b-4a57-8d15-3e419fef0fc2 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:30.300287 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c0f4ab9b-4e5b-4a57-8d15-3e419fef0fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:30.316853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4ab9b-4e5b-4a57-8d15-3e419fef0fc2 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:30.324436 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa945e-0f0d-484f-adc2-24ad16adbe30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:30.329584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa945e-0f0d-484f-adc2-24ad16adbe30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:30.329939 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa945e-0f0d-484f-adc2-24ad16adbe30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:30.330163 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa945e-0f0d-484f-adc2-24ad16adbe30 admin admin] PATH_INFO: `/v3/projects/70550ab7423c41f6a4d6a6f5d08e84b5` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:30.331805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa945e-0f0d-484f-adc2-24ad16adbe30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70550ab7423c41f6a4d6a6f5d08e84b5)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:30.335291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaa945e-0f0d-484f-adc2-24ad16adbe30 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:30.497387 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9aaa945e-0f0d-484f-adc2-24ad16adbe30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70550ab7423c41f6a4d6a6f5d08e84b5', 'request_id': 'req-9aaa945e-0f0d-484f-adc2-24ad16adbe30'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:30.505100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9aaa945e-0f0d-484f-adc2-24ad16adbe30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70550ab7423c41f6a4d6a6f5d08e84b5', 'request_id': 'req-9aaa945e-0f0d-484f-adc2-24ad16adbe30'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:30.574775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9aaa945e-0f0d-484f-adc2-24ad16adbe30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70550ab7423c41f6a4d6a6f5d08e84b5', 'request_id': 'req-9aaa945e-0f0d-484f-adc2-24ad16adbe30'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:30.587556 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9aaa945e-0f0d-484f-adc2-24ad16adbe30 admin admin] The token cache is being invalidate because project 70550ab7423c41f6a4d6a6f5d08e84b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:00:30.587904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9aaa945e-0f0d-484f-adc2-24ad16adbe30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:30.588850 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9aaa945e-0f0d-484f-adc2-24ad16adbe30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:30.590963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1194/2409] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:00:30 2022] DELETE /identity/v3/projects/70550ab7423c41f6a4d6a6f5d08e84b5 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:30.595526 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e7836-9165-4a5b-aee7-b5d79ba25a78 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:30.606610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e7836-9165-4a5b-aee7-b5d79ba25a78 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:30.672828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b326e-825f-4bd5-91c6-5af82e5f2956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:30.673896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b326e-825f-4bd5-91c6-5af82e5f2956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:30.674180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b326e-825f-4bd5-91c6-5af82e5f2956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:30.674530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b326e-825f-4bd5-91c6-5af82e5f2956 admin admin] PATH_INFO: `/v3/users/91ac861547e14ed5b944c306b71c1fd0` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:30.676722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b326e-825f-4bd5-91c6-5af82e5f2956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ac861547e14ed5b944c306b71c1fd0)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:30.679222 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23b326e-825f-4bd5-91c6-5af82e5f2956 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:30.856169 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a23b326e-825f-4bd5-91c6-5af82e5f2956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ac861547e14ed5b944c306b71c1fd0'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:30.869664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a23b326e-825f-4bd5-91c6-5af82e5f2956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ac861547e14ed5b944c306b71c1fd0'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:30.934551 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a23b326e-825f-4bd5-91c6-5af82e5f2956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ac861547e14ed5b944c306b71c1fd0'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:30.945063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a23b326e-825f-4bd5-91c6-5af82e5f2956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ac861547e14ed5b944c306b71c1fd0'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:30.950427 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a23b326e-825f-4bd5-91c6-5af82e5f2956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ac861547e14ed5b944c306b71c1fd0'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:30.952881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1216/2410] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:00:30 2022] DELETE /identity/v3/users/91ac861547e14ed5b944c306b71c1fd0 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:31.403126 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa945e-0f0d-484f-adc2-24ad16adbe30 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:31.405227 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9aaa945e-0f0d-484f-adc2-24ad16adbe30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:31.422580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaa945e-0f0d-484f-adc2-24ad16adbe30 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:31.524580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ec07c-ce3c-4461-893b-2dec161f0af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:31.525503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ec07c-ce3c-4461-893b-2dec161f0af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:31.525781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ec07c-ce3c-4461-893b-2dec161f0af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:31.526252 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ec07c-ce3c-4461-893b-2dec161f0af8 admin admin] PATH_INFO: `/v3/projects/d8e7863764a44ef794f445cfa97acbec` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:31.527924 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ec07c-ce3c-4461-893b-2dec161f0af8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8e7863764a44ef794f445cfa97acbec)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:31.530672 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ec07c-ce3c-4461-893b-2dec161f0af8 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:31.651744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-617ec07c-ce3c-4461-893b-2dec161f0af8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8e7863764a44ef794f445cfa97acbec', 'request_id': 'req-617ec07c-ce3c-4461-893b-2dec161f0af8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:31.662432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-617ec07c-ce3c-4461-893b-2dec161f0af8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8e7863764a44ef794f445cfa97acbec', 'request_id': 'req-617ec07c-ce3c-4461-893b-2dec161f0af8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:31.664679 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:31.753266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-617ec07c-ce3c-4461-893b-2dec161f0af8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8e7863764a44ef794f445cfa97acbec', 'request_id': 'req-617ec07c-ce3c-4461-893b-2dec161f0af8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:31.765308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-617ec07c-ce3c-4461-893b-2dec161f0af8 admin admin] The token cache is being invalidate because project d8e7863764a44ef794f445cfa97acbec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:00:31.765633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-617ec07c-ce3c-4461-893b-2dec161f0af8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:31.766320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-617ec07c-ce3c-4461-893b-2dec161f0af8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:31.767875 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1195/2411] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:00:31 2022] DELETE /identity/v3/projects/d8e7863764a44ef794f445cfa97acbec => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:31.774851 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25c5fe-661b-48be-ab44-4a29cb27ce27 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:31.774851 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25c5fe-661b-48be-ab44-4a29cb27ce27 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:31.774851 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d25c5fe-661b-48be-ab44-4a29cb27ce27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:31.793352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-9d25c5fe-661b-48be-ab44-4a29cb27ce27 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:31.795135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9d25c5fe-661b-48be-ab44-4a29cb27ce27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:31.873469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1217/2412] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:31.882164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e604481-9db8-4ce2-a585-d2b1616b6d91 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:31.882453 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e604481-9db8-4ce2-a585-d2b1616b6d91 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:31.882738 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e604481-9db8-4ce2-a585-d2b1616b6d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:31.902402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-8e604481-9db8-4ce2-a585-d2b1616b6d91 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:31.903546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-8e604481-9db8-4ce2-a585-d2b1616b6d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:31.912453 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1196/2413] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:31.916926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d25c5fe-661b-48be-ab44-4a29cb27ce27 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:31.927593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d25c5fe-661b-48be-ab44-4a29cb27ce27 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:31.932355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aef0e6-e755-4f86-9f55-2acb2907b9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:31.933145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aef0e6-e755-4f86-9f55-2acb2907b9ec admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:31.933584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aef0e6-e755-4f86-9f55-2acb2907b9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:31.933881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aef0e6-e755-4f86-9f55-2acb2907b9ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:31.934799 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aef0e6-e755-4f86-9f55-2acb2907b9ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:00:31.935113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aef0e6-e755-4f86-9f55-2acb2907b9ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:31.936760 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aef0e6-e755-4f86-9f55-2acb2907b9ec admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:31.953155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1218/2414] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:00:31.955067 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e604481-9db8-4ce2-a585-d2b1616b6d91 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:31.966708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e604481-9db8-4ce2-a585-d2b1616b6d91 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:31.971699 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca42d35-b2ff-4abf-8ec8-378f26906ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:31.972583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca42d35-b2ff-4abf-8ec8-378f26906ffa admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:31.973525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca42d35-b2ff-4abf-8ec8-378f26906ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:31.973805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca42d35-b2ff-4abf-8ec8-378f26906ffa admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:31.974506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca42d35-b2ff-4abf-8ec8-378f26906ffa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:31.976489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca42d35-b2ff-4abf-8ec8-378f26906ffa admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:32.080064 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1197/2415] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:00:31 2022] POST /identity/v3/projects => generated 357 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:00:32.085096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aef0e6-e755-4f86-9f55-2acb2907b9ec admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:32.096080 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aef0e6-e755-4f86-9f55-2acb2907b9ec admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:32.101421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:32.180946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb781b5-e2f3-4af0-99f0-f9b69758387f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:32.181576 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb781b5-e2f3-4af0-99f0-f9b69758387f admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:32.181862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb781b5-e2f3-4af0-99f0-f9b69758387f admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:32.182145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb781b5-e2f3-4af0-99f0-f9b69758387f admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:32.182852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb781b5-e2f3-4af0-99f0-f9b69758387f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:32.184949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb781b5-e2f3-4af0-99f0-f9b69758387f admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:32.279037 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1219/2416] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:00:32 2022] POST /identity/v3/users => generated 279 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:00:32.282931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca42d35-b2ff-4abf-8ec8-378f26906ffa admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:32.294351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca42d35-b2ff-4abf-8ec8-378f26906ffa admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:32.299356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38440aa-67ec-4d59-b8f0-89694926e711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:32.300224 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38440aa-67ec-4d59-b8f0-89694926e711 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:32.300497 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38440aa-67ec-4d59-b8f0-89694926e711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:32.300972 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38440aa-67ec-4d59-b8f0-89694926e711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:32.301732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38440aa-67ec-4d59-b8f0-89694926e711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:32.303408 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38440aa-67ec-4d59-b8f0-89694926e711 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:32.314469 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1198/2417] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:00:32.319831 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb781b5-e2f3-4af0-99f0-f9b69758387f admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:32.330276 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb781b5-e2f3-4af0-99f0-f9b69758387f admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:32.335878 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a91e9-d51f-4226-8ce4-e8c7347a7c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:32.336656 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a91e9-d51f-4226-8ce4-e8c7347a7c1a admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:32.337037 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a91e9-d51f-4226-8ce4-e8c7347a7c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:32.337392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a91e9-d51f-4226-8ce4-e8c7347a7c1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:32.338203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a91e9-d51f-4226-8ce4-e8c7347a7c1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:32.339892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a91e9-d51f-4226-8ce4-e8c7347a7c1a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:32.354953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1220/2418] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:00:32.356202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38440aa-67ec-4d59-b8f0-89694926e711 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:32.367275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38440aa-67ec-4d59-b8f0-89694926e711 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:32.372247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbefe9f-a13c-4c4b-98c6-7ccdf62aaf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:32.373184 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbefe9f-a13c-4c4b-98c6-7ccdf62aaf6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:32.373471 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbefe9f-a13c-4c4b-98c6-7ccdf62aaf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:32.373735 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbefe9f-a13c-4c4b-98c6-7ccdf62aaf6f admin admin] PATH_INFO: `/v3/projects/3786ff5bf1b241f6bfe6bffe11c5d625/users/13e47db6ee6841129865e99d8b66155f/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:32.391245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbefe9f-a13c-4c4b-98c6-7ccdf62aaf6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3786ff5bf1b241f6bfe6bffe11c5d625, user_id=13e47db6ee6841129865e99d8b66155f, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:32.394011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbefe9f-a13c-4c4b-98c6-7ccdf62aaf6f admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:32.467052 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1199/2419] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:00:32 2022] PUT /identity/v3/projects/3786ff5bf1b241f6bfe6bffe11c5d625/users/13e47db6ee6841129865e99d8b66155f/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:00:32.478653 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e00398-277f-44ee-8e17-eb95929f1469 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:32.478950 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e00398-277f-44ee-8e17-eb95929f1469 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:32.479236 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e00398-277f-44ee-8e17-eb95929f1469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:32.513036 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-70e00398-277f-44ee-8e17-eb95929f1469 None None] MFA Rules not processed for user `13e47db6ee6841129865e99d8b66155f`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:32.514726 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-70e00398-277f-44ee-8e17-eb95929f1469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:32.590261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1221/2420] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:00:32 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:32.595523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbefe9f-a13c-4c4b-98c6-7ccdf62aaf6f admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:32.673457 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbefe9f-a13c-4c4b-98c6-7ccdf62aaf6f admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:32.680567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef142ef-8886-4057-b18e-c2e29afe3104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:32.681823 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef142ef-8886-4057-b18e-c2e29afe3104 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:32.681823 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef142ef-8886-4057-b18e-c2e29afe3104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:32.681823 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef142ef-8886-4057-b18e-c2e29afe3104 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:32.682553 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef142ef-8886-4057-b18e-c2e29afe3104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:32.686043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef142ef-8886-4057-b18e-c2e29afe3104 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:32.793080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1200/2421] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:00:32 2022] POST /identity/v3/projects => generated 357 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:00:32.795520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e00398-277f-44ee-8e17-eb95929f1469 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:32.810944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e00398-277f-44ee-8e17-eb95929f1469 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:32.883680 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb9abd-d6be-4b23-979b-915f3e7b228b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:32.884477 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb9abd-d6be-4b23-979b-915f3e7b228b admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:32.884770 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb9abd-d6be-4b23-979b-915f3e7b228b admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:32.885082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb9abd-d6be-4b23-979b-915f3e7b228b admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:32.885858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb9abd-d6be-4b23-979b-915f3e7b228b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:32.887879 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb9abd-d6be-4b23-979b-915f3e7b228b admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:32.971154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1222/2422] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:00:32 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:00:32.975667 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef142ef-8886-4057-b18e-c2e29afe3104 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:32.987039 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef142ef-8886-4057-b18e-c2e29afe3104 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:32.993744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4bdd0-c093-4d30-860c-eb0cb264bd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:32.994610 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4bdd0-c093-4d30-860c-eb0cb264bd5d admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:32.994947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4bdd0-c093-4d30-860c-eb0cb264bd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:33.004450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4bdd0-c093-4d30-860c-eb0cb264bd5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:33.004450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4bdd0-c093-4d30-860c-eb0cb264bd5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:33.004450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4bdd0-c093-4d30-860c-eb0cb264bd5d admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:33.022323 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1201/2423] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:00:33.023396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb9abd-d6be-4b23-979b-915f3e7b228b admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:33.029875 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:33.033769 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb9abd-d6be-4b23-979b-915f3e7b228b admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:33.040017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216bc2d-f377-4bba-b23d-a0946e7b93d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:33.040906 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216bc2d-f377-4bba-b23d-a0946e7b93d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:33.041845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216bc2d-f377-4bba-b23d-a0946e7b93d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:33.041845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216bc2d-f377-4bba-b23d-a0946e7b93d9 admin admin] PATH_INFO: `/v3/projects/0365670a33cc4f96a7afbbec09755859/users/d36cb533b68247db9de9d582dc01b417/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:33.058047 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216bc2d-f377-4bba-b23d-a0946e7b93d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0365670a33cc4f96a7afbbec09755859, user_id=d36cb533b68247db9de9d582dc01b417, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:33.060644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216bc2d-f377-4bba-b23d-a0946e7b93d9 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:33.181422 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1223/2424] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:00:33 2022] PUT /identity/v3/projects/0365670a33cc4f96a7afbbec09755859/users/d36cb533b68247db9de9d582dc01b417/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:33.183564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4bdd0-c093-4d30-860c-eb0cb264bd5d admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:33.195983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4bdd0-c093-4d30-860c-eb0cb264bd5d admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:33.253341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:33.270165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f70a4e-0eba-4b78-b060-67b210e909e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:33.270165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f70a4e-0eba-4b78-b060-67b210e909e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:33.270165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f70a4e-0eba-4b78-b060-67b210e909e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:33.270165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f70a4e-0eba-4b78-b060-67b210e909e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:33.270638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f70a4e-0eba-4b78-b060-67b210e909e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:33.270638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f70a4e-0eba-4b78-b060-67b210e909e1 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:33.277236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1202/2425] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:00:33.281592 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216bc2d-f377-4bba-b23d-a0946e7b93d9 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:33.293621 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216bc2d-f377-4bba-b23d-a0946e7b93d9 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:33.298959 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407a646-3810-461b-86bc-a7bc37824c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:33.300074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407a646-3810-461b-86bc-a7bc37824c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:33.300520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407a646-3810-461b-86bc-a7bc37824c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:33.301708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407a646-3810-461b-86bc-a7bc37824c3d admin admin] PATH_INFO: `/v3/projects/0365670a33cc4f96a7afbbec09755859/users/d36cb533b68247db9de9d582dc01b417/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:33.305041 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407a646-3810-461b-86bc-a7bc37824c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0365670a33cc4f96a7afbbec09755859, user_id=d36cb533b68247db9de9d582dc01b417, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:33.314800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407a646-3810-461b-86bc-a7bc37824c3d admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:33.331433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1224/2426] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:00:33 2022] PUT /identity/v3/projects/0365670a33cc4f96a7afbbec09755859/users/d36cb533b68247db9de9d582dc01b417/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:33.333107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea949f-e2be-4818-b627-7fdcb6ee45a1 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:33.337726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea949f-e2be-4818-b627-7fdcb6ee45a1 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:33.337726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea949f-e2be-4818-b627-7fdcb6ee45a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:33.363859 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-fbea949f-e2be-4818-b627-7fdcb6ee45a1 None None] MFA Rules not processed for user `d36cb533b68247db9de9d582dc01b417`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:33.365358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-fbea949f-e2be-4818-b627-7fdcb6ee45a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:33.443808 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1203/2427] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:00:33 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:33.455978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407a646-3810-461b-86bc-a7bc37824c3d admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:33.458450 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2407a646-3810-461b-86bc-a7bc37824c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:33.524850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407a646-3810-461b-86bc-a7bc37824c3d admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:33.529470 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e3b50-a3bf-4356-9938-801dfefc0c3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:33.533002 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e3b50-a3bf-4356-9938-801dfefc0c3a service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:33.533002 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e3b50-a3bf-4356-9938-801dfefc0c3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:33.533002 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e3b50-a3bf-4356-9938-801dfefc0c3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:33.551214 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e3b50-a3bf-4356-9938-801dfefc0c3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:33.554142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e3b50-a3bf-4356-9938-801dfefc0c3a service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:33.568627 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1225/2428] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:00:33 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:00:33.800651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea949f-e2be-4818-b627-7fdcb6ee45a1 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:33.812296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea949f-e2be-4818-b627-7fdcb6ee45a1 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:33.817240 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e11d19-36f0-4fd3-923e-232914af62f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:33.818003 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e11d19-36f0-4fd3-923e-232914af62f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:33.821744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e11d19-36f0-4fd3-923e-232914af62f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:33.821744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e11d19-36f0-4fd3-923e-232914af62f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:33.897076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e11d19-36f0-4fd3-923e-232914af62f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:33.899069 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e11d19-36f0-4fd3-923e-232914af62f5 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:33.909817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1204/2429] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:00:33 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:00:43.562210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=39,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:43.905137 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=67 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:54.026661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aaf501-c2a6-4f54-ab2b-07ad425b54ea None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:54.027204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aaf501-c2a6-4f54-ab2b-07ad425b54ea None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:54.027534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aaf501-c2a6-4f54-ab2b-07ad425b54ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:54.049312 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-c6aaf501-c2a6-4f54-ab2b-07ad425b54ea None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:54.050554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c6aaf501-c2a6-4f54-ab2b-07ad425b54ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:54.130018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1226/2430] 10.222.0.60 () {58 vars in 967 bytes} [Wed Jun 15 11:00:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:54.139081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e11d19-36f0-4fd3-923e-232914af62f5 service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:54.141468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e3e11d19-36f0-4fd3-923e-232914af62f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:54.233716 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e11d19-36f0-4fd3-923e-232914af62f5 service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:54.239108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7cf158-19f1-4cb5-b058-8bb4f07c1d1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:54.241351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cf158-19f1-4cb5-b058-8bb4f07c1d1e service swift] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:54.241351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cf158-19f1-4cb5-b058-8bb4f07c1d1e service swift] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:54.241351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cf158-19f1-4cb5-b058-8bb4f07c1d1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:54.251427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cf158-19f1-4cb5-b058-8bb4f07c1d1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:54.252991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cf158-19f1-4cb5-b058-8bb4f07c1d1e service swift] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:54.263164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1205/2431] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 11:00:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 11:00:55.187617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aaf501-c2a6-4f54-ab2b-07ad425b54ea None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:55.276333 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aaf501-c2a6-4f54-ab2b-07ad425b54ea None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:55.282693 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852d291-3b9d-4bef-a0f6-14a0b1d8dfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:55.283685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852d291-3b9d-4bef-a0f6-14a0b1d8dfbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:55.283894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852d291-3b9d-4bef-a0f6-14a0b1d8dfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:55.284277 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852d291-3b9d-4bef-a0f6-14a0b1d8dfbe admin admin] PATH_INFO: `/v3/users/13e47db6ee6841129865e99d8b66155f` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:55.286335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852d291-3b9d-4bef-a0f6-14a0b1d8dfbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=13e47db6ee6841129865e99d8b66155f)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:55.287436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852d291-3b9d-4bef-a0f6-14a0b1d8dfbe admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:55.554614 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3852d291-3b9d-4bef-a0f6-14a0b1d8dfbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13e47db6ee6841129865e99d8b66155f'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:55.571889 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3852d291-3b9d-4bef-a0f6-14a0b1d8dfbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13e47db6ee6841129865e99d8b66155f'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:55.679750 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3852d291-3b9d-4bef-a0f6-14a0b1d8dfbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13e47db6ee6841129865e99d8b66155f'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:55.690910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3852d291-3b9d-4bef-a0f6-14a0b1d8dfbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13e47db6ee6841129865e99d8b66155f'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:55.692620 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3852d291-3b9d-4bef-a0f6-14a0b1d8dfbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13e47db6ee6841129865e99d8b66155f'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:55.703812 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1227/2432] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:00:55 2022] DELETE /identity/v3/users/13e47db6ee6841129865e99d8b66155f => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:55.705552 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9051d8f-3037-479c-abd6-461ab01e1592 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:55.706084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9051d8f-3037-479c-abd6-461ab01e1592 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:55.706084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9051d8f-3037-479c-abd6-461ab01e1592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:55.737252 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-c9051d8f-3037-479c-abd6-461ab01e1592 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:55.737252 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c9051d8f-3037-479c-abd6-461ab01e1592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:55.807694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1206/2433] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:55.817426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852d291-3b9d-4bef-a0f6-14a0b1d8dfbe admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:55.818881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-3852d291-3b9d-4bef-a0f6-14a0b1d8dfbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:55.853660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:55.890896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852d291-3b9d-4bef-a0f6-14a0b1d8dfbe admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:55.894600 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9daed-f16f-4ded-8150-959841d14eea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:55.895403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9daed-f16f-4ded-8150-959841d14eea service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:55.895604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9daed-f16f-4ded-8150-959841d14eea service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:55.895882 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9daed-f16f-4ded-8150-959841d14eea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:55.906646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9daed-f16f-4ded-8150-959841d14eea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:55.908103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9daed-f16f-4ded-8150-959841d14eea service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:55.920815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1228/2434] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:00:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:00:56.441599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9051d8f-3037-479c-abd6-461ab01e1592 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:56.442038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-c9051d8f-3037-479c-abd6-461ab01e1592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:56.455075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9051d8f-3037-479c-abd6-461ab01e1592 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:56.460335 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395074ab-8495-469c-8c4b-50b0a99690f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:56.461085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395074ab-8495-469c-8c4b-50b0a99690f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:56.461373 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395074ab-8495-469c-8c4b-50b0a99690f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:56.461653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395074ab-8495-469c-8c4b-50b0a99690f2 admin admin] PATH_INFO: `/v3/projects/3786ff5bf1b241f6bfe6bffe11c5d625` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:56.462961 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395074ab-8495-469c-8c4b-50b0a99690f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3786ff5bf1b241f6bfe6bffe11c5d625)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:56.465569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395074ab-8495-469c-8c4b-50b0a99690f2 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:56.613853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-395074ab-8495-469c-8c4b-50b0a99690f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3786ff5bf1b241f6bfe6bffe11c5d625', 'request_id': 'req-395074ab-8495-469c-8c4b-50b0a99690f2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:56.624950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-395074ab-8495-469c-8c4b-50b0a99690f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3786ff5bf1b241f6bfe6bffe11c5d625', 'request_id': 'req-395074ab-8495-469c-8c4b-50b0a99690f2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:56.713990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-395074ab-8495-469c-8c4b-50b0a99690f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3786ff5bf1b241f6bfe6bffe11c5d625', 'request_id': 'req-395074ab-8495-469c-8c4b-50b0a99690f2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:56.726176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-395074ab-8495-469c-8c4b-50b0a99690f2 admin admin] The token cache is being invalidate because project 3786ff5bf1b241f6bfe6bffe11c5d625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:00:56.726468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-395074ab-8495-469c-8c4b-50b0a99690f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:56.727130 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-395074ab-8495-469c-8c4b-50b0a99690f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:56.728678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1207/2435] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:00:56 2022] DELETE /identity/v3/projects/3786ff5bf1b241f6bfe6bffe11c5d625 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:56.733203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9daed-f16f-4ded-8150-959841d14eea service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:56.743471 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9daed-f16f-4ded-8150-959841d14eea service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:56.804132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e9f74-3dae-485c-b1ba-200e0faf2d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:56.805042 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9f74-3dae-485c-b1ba-200e0faf2d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:56.805326 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9f74-3dae-485c-b1ba-200e0faf2d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:56.806262 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e9f74-3dae-485c-b1ba-200e0faf2d59 admin admin] PATH_INFO: `/v3/users/d36cb533b68247db9de9d582dc01b417` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:56.807232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e9f74-3dae-485c-b1ba-200e0faf2d59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d36cb533b68247db9de9d582dc01b417)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:56.809986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e9f74-3dae-485c-b1ba-200e0faf2d59 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:57.030417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0f9e9f74-3dae-485c-b1ba-200e0faf2d59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd36cb533b68247db9de9d582dc01b417'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:57.042819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0f9e9f74-3dae-485c-b1ba-200e0faf2d59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd36cb533b68247db9de9d582dc01b417'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:57.106781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0f9e9f74-3dae-485c-b1ba-200e0faf2d59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd36cb533b68247db9de9d582dc01b417'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:57.115465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0f9e9f74-3dae-485c-b1ba-200e0faf2d59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd36cb533b68247db9de9d582dc01b417'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:57.116203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0f9e9f74-3dae-485c-b1ba-200e0faf2d59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd36cb533b68247db9de9d582dc01b417'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:57.123861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1229/2436] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:00:56 2022] DELETE /identity/v3/users/d36cb533b68247db9de9d582dc01b417 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:57.567312 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395074ab-8495-469c-8c4b-50b0a99690f2 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:57.568876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-395074ab-8495-469c-8c4b-50b0a99690f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:57.586187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395074ab-8495-469c-8c4b-50b0a99690f2 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:57.597813 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:57.656778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740d74a-06a1-473e-a4c6-458bdd15c3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:57.658055 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740d74a-06a1-473e-a4c6-458bdd15c3a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:57.658552 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740d74a-06a1-473e-a4c6-458bdd15c3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:57.658834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740d74a-06a1-473e-a4c6-458bdd15c3a4 admin admin] PATH_INFO: `/v3/projects/0365670a33cc4f96a7afbbec09755859` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:57.660057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740d74a-06a1-473e-a4c6-458bdd15c3a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0365670a33cc4f96a7afbbec09755859)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:57.662074 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f740d74a-06a1-473e-a4c6-458bdd15c3a4 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:57.795462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f740d74a-06a1-473e-a4c6-458bdd15c3a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0365670a33cc4f96a7afbbec09755859', 'request_id': 'req-f740d74a-06a1-473e-a4c6-458bdd15c3a4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:57.803013 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f740d74a-06a1-473e-a4c6-458bdd15c3a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0365670a33cc4f96a7afbbec09755859', 'request_id': 'req-f740d74a-06a1-473e-a4c6-458bdd15c3a4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:57.886297 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f740d74a-06a1-473e-a4c6-458bdd15c3a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0365670a33cc4f96a7afbbec09755859', 'request_id': 'req-f740d74a-06a1-473e-a4c6-458bdd15c3a4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:57.897686 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f740d74a-06a1-473e-a4c6-458bdd15c3a4 admin admin] The token cache is being invalidate because project 0365670a33cc4f96a7afbbec09755859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:00:57.897978 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f740d74a-06a1-473e-a4c6-458bdd15c3a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:57.898621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f740d74a-06a1-473e-a4c6-458bdd15c3a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:00:57.900158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1208/2437] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:00:57 2022] DELETE /identity/v3/projects/0365670a33cc4f96a7afbbec09755859 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:57.906449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df823f-323d-43e8-adcc-698a12c09198 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:57.906992 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df823f-323d-43e8-adcc-698a12c09198 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:57.907144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df823f-323d-43e8-adcc-698a12c09198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:57.925709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-c9df823f-323d-43e8-adcc-698a12c09198 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:57.926903 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c9df823f-323d-43e8-adcc-698a12c09198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:57.971256 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:57.989362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1230/2438] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:58.000166 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243a7ca-c0c3-426e-9bc7-e348b8c9ff8b None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:58.000460 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243a7ca-c0c3-426e-9bc7-e348b8c9ff8b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:58.000743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243a7ca-c0c3-426e-9bc7-e348b8c9ff8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:58.019946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-7243a7ca-c0c3-426e-9bc7-e348b8c9ff8b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:58.021095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-7243a7ca-c0c3-426e-9bc7-e348b8c9ff8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:58.029682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1209/2439] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:58.034570 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df823f-323d-43e8-adcc-698a12c09198 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:58.044948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df823f-323d-43e8-adcc-698a12c09198 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:58.050720 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154a3d6-727b-47c6-879b-7d8ef2cd41f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:58.065294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154a3d6-727b-47c6-879b-7d8ef2cd41f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:58.065294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154a3d6-727b-47c6-879b-7d8ef2cd41f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:58.065294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154a3d6-727b-47c6-879b-7d8ef2cd41f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:58.065294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154a3d6-727b-47c6-879b-7d8ef2cd41f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:00:58.065294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154a3d6-727b-47c6-879b-7d8ef2cd41f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:58.065294 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154a3d6-727b-47c6-879b-7d8ef2cd41f1 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:58.067150 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1231/2440] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:00:58.072022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243a7ca-c0c3-426e-9bc7-e348b8c9ff8b None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:58.081457 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243a7ca-c0c3-426e-9bc7-e348b8c9ff8b None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:58.086491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43ba5b-397c-4688-a06d-40781b7aed2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:58.087158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43ba5b-397c-4688-a06d-40781b7aed2f admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:58.087444 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43ba5b-397c-4688-a06d-40781b7aed2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:58.087720 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43ba5b-397c-4688-a06d-40781b7aed2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:58.088479 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43ba5b-397c-4688-a06d-40781b7aed2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:58.091092 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43ba5b-397c-4688-a06d-40781b7aed2f admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:58.333310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1210/2441] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:00:58 2022] POST /identity/v3/projects => generated 381 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:00:58.368525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154a3d6-727b-47c6-879b-7d8ef2cd41f1 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:58.368525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154a3d6-727b-47c6-879b-7d8ef2cd41f1 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:58.404666 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed8889-a6d0-458e-97a2-f4e12857485f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:58.405372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed8889-a6d0-458e-97a2-f4e12857485f admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:58.405658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed8889-a6d0-458e-97a2-f4e12857485f admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:58.405935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed8889-a6d0-458e-97a2-f4e12857485f admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:58.406627 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed8889-a6d0-458e-97a2-f4e12857485f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:58.408581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed8889-a6d0-458e-97a2-f4e12857485f admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:58.537342 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1232/2442] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:00:58 2022] POST /identity/v3/users => generated 291 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:00:58.541817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43ba5b-397c-4688-a06d-40781b7aed2f admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:58.553322 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43ba5b-397c-4688-a06d-40781b7aed2f admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:58.557927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3e5f2-851c-4e51-8ca4-951060b8dc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:58.559423 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3e5f2-851c-4e51-8ca4-951060b8dc36 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:58.559808 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3e5f2-851c-4e51-8ca4-951060b8dc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:58.560084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3e5f2-851c-4e51-8ca4-951060b8dc36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:58.560821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3e5f2-851c-4e51-8ca4-951060b8dc36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:58.562474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3e5f2-851c-4e51-8ca4-951060b8dc36 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:58.575502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1211/2443] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:00:58.580945 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed8889-a6d0-458e-97a2-f4e12857485f admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:58.591926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed8889-a6d0-458e-97a2-f4e12857485f admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:58.596743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afe57a-c06d-4b58-bc42-b171d70353e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:58.597444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afe57a-c06d-4b58-bc42-b171d70353e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:58.597736 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afe57a-c06d-4b58-bc42-b171d70353e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:58.598030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afe57a-c06d-4b58-bc42-b171d70353e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:58.598763 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afe57a-c06d-4b58-bc42-b171d70353e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:58.600377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afe57a-c06d-4b58-bc42-b171d70353e0 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:58.611115 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1233/2444] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:00:58.615366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3e5f2-851c-4e51-8ca4-951060b8dc36 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:58.626210 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3e5f2-851c-4e51-8ca4-951060b8dc36 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:58.630861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8960b47-209c-4f2c-8eca-659b2cc6b335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:58.631792 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8960b47-209c-4f2c-8eca-659b2cc6b335 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:58.632078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8960b47-209c-4f2c-8eca-659b2cc6b335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:58.632359 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8960b47-209c-4f2c-8eca-659b2cc6b335 admin admin] PATH_INFO: `/v3/projects/bc95d8e517bc42a4b99c18d4de326c2f/users/5212ff5e8b1b45b6b9192be1a43fa2a1/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:58.647633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8960b47-209c-4f2c-8eca-659b2cc6b335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc95d8e517bc42a4b99c18d4de326c2f, user_id=5212ff5e8b1b45b6b9192be1a43fa2a1, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:58.650382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8960b47-209c-4f2c-8eca-659b2cc6b335 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:58.755010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1212/2445] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:00:58 2022] PUT /identity/v3/projects/bc95d8e517bc42a4b99c18d4de326c2f/users/5212ff5e8b1b45b6b9192be1a43fa2a1/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:58.766283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35441b0e-981e-441d-bb4f-94e8c765226f None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:58.766572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35441b0e-981e-441d-bb4f-94e8c765226f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:58.766850 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35441b0e-981e-441d-bb4f-94e8c765226f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:58.796791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-35441b0e-981e-441d-bb4f-94e8c765226f None None] MFA Rules not processed for user `5212ff5e8b1b45b6b9192be1a43fa2a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:58.798286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-35441b0e-981e-441d-bb4f-94e8c765226f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:58.872102 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1234/2446] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:00:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:58.876882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8960b47-209c-4f2c-8eca-659b2cc6b335 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:58.930693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:58.952336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8960b47-209c-4f2c-8eca-659b2cc6b335 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:58.957651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb1dbb-a945-4b51-b413-482ac250adb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:58.958323 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb1dbb-a945-4b51-b413-482ac250adb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:58.958607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb1dbb-a945-4b51-b413-482ac250adb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:58.958885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb1dbb-a945-4b51-b413-482ac250adb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:58.959645 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb1dbb-a945-4b51-b413-482ac250adb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:58.961640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb1dbb-a945-4b51-b413-482ac250adb6 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:59.088441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1213/2447] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:00:58 2022] POST /identity/v3/projects => generated 381 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:00:59.092628 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35441b0e-981e-441d-bb4f-94e8c765226f None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:59.104307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35441b0e-981e-441d-bb4f-94e8c765226f None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:59.149504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:00:59.168644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e2f7b-9b04-4b9d-adfc-1d805635be30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:59.169441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e2f7b-9b04-4b9d-adfc-1d805635be30 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:59.169729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e2f7b-9b04-4b9d-adfc-1d805635be30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:59.170005 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e2f7b-9b04-4b9d-adfc-1d805635be30 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:59.170716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e2f7b-9b04-4b9d-adfc-1d805635be30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:59.172666 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e2f7b-9b04-4b9d-adfc-1d805635be30 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:59.305547 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1235/2448] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:00:59 2022] POST /identity/v3/users => generated 297 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:00:59.307266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb1dbb-a945-4b51-b413-482ac250adb6 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:59.318300 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb1dbb-a945-4b51-b413-482ac250adb6 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:59.323160 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11655c1-3589-4f44-909a-a196eddf184c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:59.324236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11655c1-3589-4f44-909a-a196eddf184c admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:59.326029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11655c1-3589-4f44-909a-a196eddf184c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:59.326711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11655c1-3589-4f44-909a-a196eddf184c admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:59.328075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11655c1-3589-4f44-909a-a196eddf184c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:59.330001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11655c1-3589-4f44-909a-a196eddf184c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:59.343100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1214/2449] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:00:59.347626 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e2f7b-9b04-4b9d-adfc-1d805635be30 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:59.359806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e2f7b-9b04-4b9d-adfc-1d805635be30 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:59.364914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b696f-65d1-4f0d-8df3-4a1d10e333fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:59.366981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b696f-65d1-4f0d-8df3-4a1d10e333fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:59.366981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b696f-65d1-4f0d-8df3-4a1d10e333fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:59.366981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b696f-65d1-4f0d-8df3-4a1d10e333fa admin admin] PATH_INFO: `/v3/projects/961a30c0430f43bb8463d0e3aafd1bcd/users/ac4faa49ad96491a865312cfa60f67c8/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:59.385298 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b696f-65d1-4f0d-8df3-4a1d10e333fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=961a30c0430f43bb8463d0e3aafd1bcd, user_id=ac4faa49ad96491a865312cfa60f67c8, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:59.388725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b696f-65d1-4f0d-8df3-4a1d10e333fa admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:59.457015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1236/2450] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:00:59 2022] PUT /identity/v3/projects/961a30c0430f43bb8463d0e3aafd1bcd/users/ac4faa49ad96491a865312cfa60f67c8/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:59.461401 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11655c1-3589-4f44-909a-a196eddf184c admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:59.473195 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11655c1-3589-4f44-909a-a196eddf184c admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:59.535725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b6d4a-b8d7-4693-9ded-5d3f6ec8edad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:59.540912 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b6d4a-b8d7-4693-9ded-5d3f6ec8edad admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:59.540912 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b6d4a-b8d7-4693-9ded-5d3f6ec8edad admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:59.540912 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b6d4a-b8d7-4693-9ded-5d3f6ec8edad admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:59.540912 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b6d4a-b8d7-4693-9ded-5d3f6ec8edad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:59.540912 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b6d4a-b8d7-4693-9ded-5d3f6ec8edad admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:59.550897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1215/2451] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:00:59.555016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b696f-65d1-4f0d-8df3-4a1d10e333fa admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:59.566287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b696f-65d1-4f0d-8df3-4a1d10e333fa admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:59.571044 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32b6e6-901d-4fcd-a9a8-31eec755d8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:59.571989 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32b6e6-901d-4fcd-a9a8-31eec755d8a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:59.572272 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32b6e6-901d-4fcd-a9a8-31eec755d8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:59.572548 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32b6e6-901d-4fcd-a9a8-31eec755d8a3 admin admin] PATH_INFO: `/v3/projects/961a30c0430f43bb8463d0e3aafd1bcd/users/ac4faa49ad96491a865312cfa60f67c8/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:59.575265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32b6e6-901d-4fcd-a9a8-31eec755d8a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=961a30c0430f43bb8463d0e3aafd1bcd, user_id=ac4faa49ad96491a865312cfa60f67c8, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:59.578066 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32b6e6-901d-4fcd-a9a8-31eec755d8a3 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:59.589424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1237/2452] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:00:59 2022] PUT /identity/v3/projects/961a30c0430f43bb8463d0e3aafd1bcd/users/ac4faa49ad96491a865312cfa60f67c8/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:00:59.602074 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de754941-1390-4337-a9a1-85c08d861890 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:59.602407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de754941-1390-4337-a9a1-85c08d861890 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:59.602808 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de754941-1390-4337-a9a1-85c08d861890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:59.632542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-de754941-1390-4337-a9a1-85c08d861890 None None] MFA Rules not processed for user `ac4faa49ad96491a865312cfa60f67c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:00:59.633799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-de754941-1390-4337-a9a1-85c08d861890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:59.712296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1216/2453] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:00:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:00:59.726364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32b6e6-901d-4fcd-a9a8-31eec755d8a3 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:00:59.728490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-fe32b6e6-901d-4fcd-a9a8-31eec755d8a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:00:59.792733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32b6e6-901d-4fcd-a9a8-31eec755d8a3 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:00:59.796997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3592d17-d645-432d-bf14-656ab8532212 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:00:59.797659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3592d17-d645-432d-bf14-656ab8532212 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:00:59.797944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3592d17-d645-432d-bf14-656ab8532212 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:00:59.798244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3592d17-d645-432d-bf14-656ab8532212 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:00:59.808033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3592d17-d645-432d-bf14-656ab8532212 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:00:59.810434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3592d17-d645-432d-bf14-656ab8532212 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:00:59.820504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1238/2454] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:00:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:01:00.886697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de754941-1390-4337-a9a1-85c08d861890 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:00.897156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de754941-1390-4337-a9a1-85c08d861890 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:00.901897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679c7f6-d69e-4b63-a820-e89db2d36b99 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4faa49ad96491a865312cfa60f67c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '961a30c0430f43bb8463d0e3aafd1bcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:00.903122 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679c7f6-d69e-4b63-a820-e89db2d36b99 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:00.903630 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679c7f6-d69e-4b63-a820-e89db2d36b99 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:00.903922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679c7f6-d69e-4b63-a820-e89db2d36b99 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:00.904679 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679c7f6-d69e-4b63-a820-e89db2d36b99 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:01:00.905018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679c7f6-d69e-4b63-a820-e89db2d36b99 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:00.906774 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679c7f6-d69e-4b63-a820-e89db2d36b99 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:00.924689 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1217/2455] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:01:00.925589 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3592d17-d645-432d-bf14-656ab8532212 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:00.935741 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3592d17-d645-432d-bf14-656ab8532212 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:00.940844 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e38fa-60da-40f7-a9b0-93335b5c4e51 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4faa49ad96491a865312cfa60f67c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '961a30c0430f43bb8463d0e3aafd1bcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:00.941551 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e38fa-60da-40f7-a9b0-93335b5c4e51 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:00.941851 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e38fa-60da-40f7-a9b0-93335b5c4e51 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:00.942135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e38fa-60da-40f7-a9b0-93335b5c4e51 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:00.942860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e38fa-60da-40f7-a9b0-93335b5c4e51 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:00.944822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e38fa-60da-40f7-a9b0-93335b5c4e51 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:01.049784 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1239/2456] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:01:00 2022] POST /identity/v3/projects => generated 354 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:01:01.054346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679c7f6-d69e-4b63-a820-e89db2d36b99 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:01.065648 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679c7f6-d69e-4b63-a820-e89db2d36b99 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:01.087264 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:01.137211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d58e7a-1c64-41b5-99a0-bfe7c1483fc8 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4faa49ad96491a865312cfa60f67c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '961a30c0430f43bb8463d0e3aafd1bcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:01.137937 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d58e7a-1c64-41b5-99a0-bfe7c1483fc8 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:01.138185 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d58e7a-1c64-41b5-99a0-bfe7c1483fc8 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:01.138449 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d58e7a-1c64-41b5-99a0-bfe7c1483fc8 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:01.139201 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d58e7a-1c64-41b5-99a0-bfe7c1483fc8 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:01:01.139513 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d58e7a-1c64-41b5-99a0-bfe7c1483fc8 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:01.141184 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d58e7a-1c64-41b5-99a0-bfe7c1483fc8 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:01.157990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1218/2457] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:01:01.206857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e38fa-60da-40f7-a9b0-93335b5c4e51 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:01.218695 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e38fa-60da-40f7-a9b0-93335b5c4e51 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:01.224268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd84aa-2782-49a8-917e-2e114d179335 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac4faa49ad96491a865312cfa60f67c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '961a30c0430f43bb8463d0e3aafd1bcd', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:01.225208 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd84aa-2782-49a8-917e-2e114d179335 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:01.225498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd84aa-2782-49a8-917e-2e114d179335 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:01.225900 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd84aa-2782-49a8-917e-2e114d179335 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] PATH_INFO: `/v3/projects/c731cc6ecbed42b1aa428e72fb79995c` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:01.227019 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd84aa-2782-49a8-917e-2e114d179335 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c731cc6ecbed42b1aa428e72fb79995c)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:01.228978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd84aa-2782-49a8-917e-2e114d179335 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:01.337210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a5cd84aa-2782-49a8-917e-2e114d179335 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c731cc6ecbed42b1aa428e72fb79995c', 'request_id': 'req-a5cd84aa-2782-49a8-917e-2e114d179335'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:01.345229 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a5cd84aa-2782-49a8-917e-2e114d179335 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c731cc6ecbed42b1aa428e72fb79995c', 'request_id': 'req-a5cd84aa-2782-49a8-917e-2e114d179335'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:01.351651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a5cd84aa-2782-49a8-917e-2e114d179335 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c731cc6ecbed42b1aa428e72fb79995c', 'request_id': 'req-a5cd84aa-2782-49a8-917e-2e114d179335'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:01.430741 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:01.438266 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a5cd84aa-2782-49a8-917e-2e114d179335 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] The token cache is being invalidate because project c731cc6ecbed42b1aa428e72fb79995c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:01:01.445345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a5cd84aa-2782-49a8-917e-2e114d179335 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:01.445345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a5cd84aa-2782-49a8-917e-2e114d179335 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:01.445345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1240/2458] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:01:01 2022] DELETE /identity/v3/projects/c731cc6ecbed42b1aa428e72fb79995c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:02.249354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d58e7a-1c64-41b5-99a0-bfe7c1483fc8 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:02.251269 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-41d58e7a-1c64-41b5-99a0-bfe7c1483fc8 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:02.332139 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d58e7a-1c64-41b5-99a0-bfe7c1483fc8 tempest-VolumeQuotaClassesTest-1626753096 tempest-VolumeQuotaClassesTest-1626753096-project-admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:02.393973 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc077a-95cc-41c8-83d9-169e53c01b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:02.394899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc077a-95cc-41c8-83d9-169e53c01b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:02.395185 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc077a-95cc-41c8-83d9-169e53c01b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:02.395488 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc077a-95cc-41c8-83d9-169e53c01b6b admin admin] PATH_INFO: `/v3/users/5212ff5e8b1b45b6b9192be1a43fa2a1` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:02.396706 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc077a-95cc-41c8-83d9-169e53c01b6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5212ff5e8b1b45b6b9192be1a43fa2a1)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:02.398715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc077a-95cc-41c8-83d9-169e53c01b6b admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:02.629469 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-becc077a-95cc-41c8-83d9-169e53c01b6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5212ff5e8b1b45b6b9192be1a43fa2a1'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:02.712552 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-becc077a-95cc-41c8-83d9-169e53c01b6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5212ff5e8b1b45b6b9192be1a43fa2a1'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:02.713218 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-becc077a-95cc-41c8-83d9-169e53c01b6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5212ff5e8b1b45b6b9192be1a43fa2a1'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:02.713823 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-becc077a-95cc-41c8-83d9-169e53c01b6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5212ff5e8b1b45b6b9192be1a43fa2a1'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:02.724245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:02.728730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-becc077a-95cc-41c8-83d9-169e53c01b6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5212ff5e8b1b45b6b9192be1a43fa2a1'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:02.741298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1219/2459] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:01:02 2022] DELETE /identity/v3/users/5212ff5e8b1b45b6b9192be1a43fa2a1 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:02.746020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee04f8-4007-463e-b3e4-e513783a7527 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:02.746577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee04f8-4007-463e-b3e4-e513783a7527 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:02.746577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee04f8-4007-463e-b3e4-e513783a7527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:02.765754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-76ee04f8-4007-463e-b3e4-e513783a7527 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:01:02.766972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-76ee04f8-4007-463e-b3e4-e513783a7527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:02.831067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1241/2460] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:01:02.845631 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc077a-95cc-41c8-83d9-169e53c01b6b admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:02.845631 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-becc077a-95cc-41c8-83d9-169e53c01b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:02.905633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc077a-95cc-41c8-83d9-169e53c01b6b admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:02.909374 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c5f82-42af-4af4-9e82-1911d9194c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:02.910032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c5f82-42af-4af4-9e82-1911d9194c7c service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:02.910032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c5f82-42af-4af4-9e82-1911d9194c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:02.910601 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c5f82-42af-4af4-9e82-1911d9194c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:02.920166 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c5f82-42af-4af4-9e82-1911d9194c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:02.921883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c5f82-42af-4af4-9e82-1911d9194c7c service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:02.932038 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1220/2461] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:01:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:01:03.382545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee04f8-4007-463e-b3e4-e513783a7527 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:03.383836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-76ee04f8-4007-463e-b3e4-e513783a7527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:03.396383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee04f8-4007-463e-b3e4-e513783a7527 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:03.401283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b393d-570c-4fd9-9136-0bc9146906f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:03.402243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b393d-570c-4fd9-9136-0bc9146906f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:03.402594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b393d-570c-4fd9-9136-0bc9146906f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:03.403018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b393d-570c-4fd9-9136-0bc9146906f5 admin admin] PATH_INFO: `/v3/projects/bc95d8e517bc42a4b99c18d4de326c2f` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:03.404242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b393d-570c-4fd9-9136-0bc9146906f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc95d8e517bc42a4b99c18d4de326c2f)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:03.406207 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b393d-570c-4fd9-9136-0bc9146906f5 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:03.545741 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-de1b393d-570c-4fd9-9136-0bc9146906f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc95d8e517bc42a4b99c18d4de326c2f', 'request_id': 'req-de1b393d-570c-4fd9-9136-0bc9146906f5'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:03.552566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-de1b393d-570c-4fd9-9136-0bc9146906f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc95d8e517bc42a4b99c18d4de326c2f', 'request_id': 'req-de1b393d-570c-4fd9-9136-0bc9146906f5'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:03.559113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-de1b393d-570c-4fd9-9136-0bc9146906f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc95d8e517bc42a4b99c18d4de326c2f', 'request_id': 'req-de1b393d-570c-4fd9-9136-0bc9146906f5'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:03.639677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-de1b393d-570c-4fd9-9136-0bc9146906f5 admin admin] The token cache is being invalidate because project bc95d8e517bc42a4b99c18d4de326c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:01:03.639972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-de1b393d-570c-4fd9-9136-0bc9146906f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:03.640614 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-de1b393d-570c-4fd9-9136-0bc9146906f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:03.642648 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1242/2462] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:01:03 2022] DELETE /identity/v3/projects/bc95d8e517bc42a4b99c18d4de326c2f => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:03.646805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c5f82-42af-4af4-9e82-1911d9194c7c service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:03.657137 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c5f82-42af-4af4-9e82-1911d9194c7c service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:03.719557 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f15aec-39cf-42a9-a4ef-3e7af01a43c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:03.720435 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f15aec-39cf-42a9-a4ef-3e7af01a43c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:03.720715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f15aec-39cf-42a9-a4ef-3e7af01a43c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:03.721019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f15aec-39cf-42a9-a4ef-3e7af01a43c7 admin admin] PATH_INFO: `/v3/users/ac4faa49ad96491a865312cfa60f67c8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:03.722217 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f15aec-39cf-42a9-a4ef-3e7af01a43c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac4faa49ad96491a865312cfa60f67c8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:03.724139 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f15aec-39cf-42a9-a4ef-3e7af01a43c7 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:03.927277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b4f15aec-39cf-42a9-a4ef-3e7af01a43c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac4faa49ad96491a865312cfa60f67c8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:03.998791 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b4f15aec-39cf-42a9-a4ef-3e7af01a43c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac4faa49ad96491a865312cfa60f67c8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:03.999405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b4f15aec-39cf-42a9-a4ef-3e7af01a43c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac4faa49ad96491a865312cfa60f67c8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:04.000007 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b4f15aec-39cf-42a9-a4ef-3e7af01a43c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac4faa49ad96491a865312cfa60f67c8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:04.012334 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-b4f15aec-39cf-42a9-a4ef-3e7af01a43c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac4faa49ad96491a865312cfa60f67c8'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:04.026065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1221/2463] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:01:03 2022] DELETE /identity/v3/users/ac4faa49ad96491a865312cfa60f67c8 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:04.441464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b393d-570c-4fd9-9136-0bc9146906f5 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:04.443312 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-de1b393d-570c-4fd9-9136-0bc9146906f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:04.459401 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b393d-570c-4fd9-9136-0bc9146906f5 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:04.521978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2935949-58ef-4e12-a89e-279ec319effa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:04.522839 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2935949-58ef-4e12-a89e-279ec319effa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:04.523188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2935949-58ef-4e12-a89e-279ec319effa admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:04.523386 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2935949-58ef-4e12-a89e-279ec319effa admin admin] PATH_INFO: `/v3/projects/961a30c0430f43bb8463d0e3aafd1bcd` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:04.524630 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2935949-58ef-4e12-a89e-279ec319effa admin admin] RBAC: Authorizing `identity:delete_project(project_id=961a30c0430f43bb8463d0e3aafd1bcd)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:04.526670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2935949-58ef-4e12-a89e-279ec319effa admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:04.553422 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:04.668740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d2935949-58ef-4e12-a89e-279ec319effa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '961a30c0430f43bb8463d0e3aafd1bcd', 'request_id': 'req-d2935949-58ef-4e12-a89e-279ec319effa'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:04.675091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d2935949-58ef-4e12-a89e-279ec319effa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '961a30c0430f43bb8463d0e3aafd1bcd', 'request_id': 'req-d2935949-58ef-4e12-a89e-279ec319effa'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:04.680704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d2935949-58ef-4e12-a89e-279ec319effa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '961a30c0430f43bb8463d0e3aafd1bcd', 'request_id': 'req-d2935949-58ef-4e12-a89e-279ec319effa'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:04.742227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d2935949-58ef-4e12-a89e-279ec319effa admin admin] The token cache is being invalidate because project 961a30c0430f43bb8463d0e3aafd1bcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:01:04.742483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d2935949-58ef-4e12-a89e-279ec319effa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:04.743070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-d2935949-58ef-4e12-a89e-279ec319effa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:04.744633 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1243/2464] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:01:04 2022] DELETE /identity/v3/projects/961a30c0430f43bb8463d0e3aafd1bcd => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:04.750298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92df26-8bf6-4e92-ae8f-d912f27903cb None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:04.750845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92df26-8bf6-4e92-ae8f-d912f27903cb None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:04.750845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92df26-8bf6-4e92-ae8f-d912f27903cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:04.759012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:04.770321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-1f92df26-8bf6-4e92-ae8f-d912f27903cb None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:01:04.772471 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1f92df26-8bf6-4e92-ae8f-d912f27903cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:04.832575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1222/2465] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:01:04.842319 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72446e-ef9c-4810-bad9-911d9b25d43a None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:04.842626 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72446e-ef9c-4810-bad9-911d9b25d43a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:04.842913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72446e-ef9c-4810-bad9-911d9b25d43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:04.862686 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-bb72446e-ef9c-4810-bad9-911d9b25d43a None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:01:04.864218 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-bb72446e-ef9c-4810-bad9-911d9b25d43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:04.873403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1244/2466] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:01:04.878223 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92df26-8bf6-4e92-ae8f-d912f27903cb None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:04.891213 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92df26-8bf6-4e92-ae8f-d912f27903cb None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:04.895937 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b3199-f603-4ad0-bd07-fff0ef849339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:04.896603 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b3199-f603-4ad0-bd07-fff0ef849339 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:04.896887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b3199-f603-4ad0-bd07-fff0ef849339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:04.897197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b3199-f603-4ad0-bd07-fff0ef849339 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:04.897931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b3199-f603-4ad0-bd07-fff0ef849339 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:01:04.898240 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b3199-f603-4ad0-bd07-fff0ef849339 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:04.899857 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b3199-f603-4ad0-bd07-fff0ef849339 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:04.911580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1223/2467] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:01:04.916090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72446e-ef9c-4810-bad9-911d9b25d43a None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:04.925904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72446e-ef9c-4810-bad9-911d9b25d43a None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:04.930566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e19fa9-9761-401d-a63e-270a33291302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:04.931208 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e19fa9-9761-401d-a63e-270a33291302 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:04.931497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e19fa9-9761-401d-a63e-270a33291302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:04.931770 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e19fa9-9761-401d-a63e-270a33291302 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:04.932462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e19fa9-9761-401d-a63e-270a33291302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:04.935173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e19fa9-9761-401d-a63e-270a33291302 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:05.041377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1245/2468] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:01:04 2022] POST /identity/v3/projects => generated 393 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:01:05.046299 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b3199-f603-4ad0-bd07-fff0ef849339 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:05.056419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b3199-f603-4ad0-bd07-fff0ef849339 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:05.115583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77514d86-4b66-447d-97bb-55327ea98330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:05.116398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77514d86-4b66-447d-97bb-55327ea98330 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:05.116692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77514d86-4b66-447d-97bb-55327ea98330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:05.117007 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77514d86-4b66-447d-97bb-55327ea98330 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:05.117705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77514d86-4b66-447d-97bb-55327ea98330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:05.119656 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77514d86-4b66-447d-97bb-55327ea98330 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:05.205676 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1224/2469] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:01:05 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:01:05.209996 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e19fa9-9761-401d-a63e-270a33291302 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:05.221356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e19fa9-9761-401d-a63e-270a33291302 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:05.226087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a264a3b-ec00-41f5-a85c-a6ec002e4065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:05.226752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a264a3b-ec00-41f5-a85c-a6ec002e4065 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:05.227039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a264a3b-ec00-41f5-a85c-a6ec002e4065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:05.227320 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a264a3b-ec00-41f5-a85c-a6ec002e4065 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:05.228053 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a264a3b-ec00-41f5-a85c-a6ec002e4065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:05.229968 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a264a3b-ec00-41f5-a85c-a6ec002e4065 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:05.241095 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1246/2470] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:01:05.245145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77514d86-4b66-447d-97bb-55327ea98330 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:05.255377 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77514d86-4b66-447d-97bb-55327ea98330 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:05.259896 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18192f-3670-461b-944f-e66b380acadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:05.260547 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18192f-3670-461b-944f-e66b380acadc admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:05.260827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18192f-3670-461b-944f-e66b380acadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:05.261132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18192f-3670-461b-944f-e66b380acadc admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:05.261860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18192f-3670-461b-944f-e66b380acadc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:05.263577 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18192f-3670-461b-944f-e66b380acadc admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:05.273354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1225/2471] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:01:05.277607 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a264a3b-ec00-41f5-a85c-a6ec002e4065 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:05.288535 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a264a3b-ec00-41f5-a85c-a6ec002e4065 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:05.294898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0552b-1e2b-4030-8249-2b8d44a0a9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:05.295230 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0552b-1e2b-4030-8249-2b8d44a0a9f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:05.295514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0552b-1e2b-4030-8249-2b8d44a0a9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:05.295795 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0552b-1e2b-4030-8249-2b8d44a0a9f9 admin admin] PATH_INFO: `/v3/projects/ed000e041ab74229b0944ef9efa4ddbf/users/452b9ca5536d41d192451fb64c9250a4/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:05.310739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0552b-1e2b-4030-8249-2b8d44a0a9f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed000e041ab74229b0944ef9efa4ddbf, user_id=452b9ca5536d41d192451fb64c9250a4, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:05.313494 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0552b-1e2b-4030-8249-2b8d44a0a9f9 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:05.394805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1247/2472] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:01:05 2022] PUT /identity/v3/projects/ed000e041ab74229b0944ef9efa4ddbf/users/452b9ca5536d41d192451fb64c9250a4/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:05.410486 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3b00f-8ecd-4c6d-9f50-df1082d7a0cb None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:05.410770 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3b00f-8ecd-4c6d-9f50-df1082d7a0cb None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:05.411050 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3b00f-8ecd-4c6d-9f50-df1082d7a0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:05.441745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-04a3b00f-8ecd-4c6d-9f50-df1082d7a0cb None None] MFA Rules not processed for user `452b9ca5536d41d192451fb64c9250a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:01:05.442970 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-04a3b00f-8ecd-4c6d-9f50-df1082d7a0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:05.508946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:05.513379 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1226/2473] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:01:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:01:05.517879 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0552b-1e2b-4030-8249-2b8d44a0a9f9 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:05.583333 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0552b-1e2b-4030-8249-2b8d44a0a9f9 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:05.588436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f64ef-c41c-4e90-b55d-6d4655971bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:05.589129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f64ef-c41c-4e90-b55d-6d4655971bfa admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:05.589422 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f64ef-c41c-4e90-b55d-6d4655971bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:05.589697 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f64ef-c41c-4e90-b55d-6d4655971bfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:05.590391 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f64ef-c41c-4e90-b55d-6d4655971bfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:05.593304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f64ef-c41c-4e90-b55d-6d4655971bfa admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:05.680160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1248/2474] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:01:05 2022] POST /identity/v3/projects => generated 391 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:01:05.684264 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3b00f-8ecd-4c6d-9f50-df1082d7a0cb None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:05.695712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3b00f-8ecd-4c6d-9f50-df1082d7a0cb None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:05.756167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9a15b-7078-446c-90a2-15d49704be7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:05.756897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9a15b-7078-446c-90a2-15d49704be7b admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:05.757158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9a15b-7078-446c-90a2-15d49704be7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:05.757438 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9a15b-7078-446c-90a2-15d49704be7b admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:05.758179 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9a15b-7078-446c-90a2-15d49704be7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:05.760135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9a15b-7078-446c-90a2-15d49704be7b admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:05.852156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1227/2475] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:01:05 2022] POST /identity/v3/users => generated 302 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:01:05.856713 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f64ef-c41c-4e90-b55d-6d4655971bfa admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:05.867847 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f64ef-c41c-4e90-b55d-6d4655971bfa admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:05.872865 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad60040-366f-47dd-b236-459e91c18c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:05.873575 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad60040-366f-47dd-b236-459e91c18c36 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:05.873856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad60040-366f-47dd-b236-459e91c18c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:05.874128 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad60040-366f-47dd-b236-459e91c18c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:05.874894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad60040-366f-47dd-b236-459e91c18c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:05.876536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad60040-366f-47dd-b236-459e91c18c36 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:05.888981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1249/2476] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:01:05.893452 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9a15b-7078-446c-90a2-15d49704be7b admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:05.904930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9a15b-7078-446c-90a2-15d49704be7b admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:05.909822 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ca1ae-953c-4b1d-8340-0d2964a52c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:05.910723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ca1ae-953c-4b1d-8340-0d2964a52c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:05.911003 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ca1ae-953c-4b1d-8340-0d2964a52c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:05.911280 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ca1ae-953c-4b1d-8340-0d2964a52c8b admin admin] PATH_INFO: `/v3/projects/2cef7f8b1e6948bcb185caa2d8ec8bb3/users/2333f60db9b2482e9cdc036f242c5485/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:05.927897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ca1ae-953c-4b1d-8340-0d2964a52c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cef7f8b1e6948bcb185caa2d8ec8bb3, user_id=2333f60db9b2482e9cdc036f242c5485, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:05.930693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ca1ae-953c-4b1d-8340-0d2964a52c8b admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:06.040098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1228/2477] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:01:05 2022] PUT /identity/v3/projects/2cef7f8b1e6948bcb185caa2d8ec8bb3/users/2333f60db9b2482e9cdc036f242c5485/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:01:06.044395 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad60040-366f-47dd-b236-459e91c18c36 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:06.055582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad60040-366f-47dd-b236-459e91c18c36 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:06.077041 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:06.117936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f104d8c-5c79-4fe2-ae67-4349cc28c608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:06.118531 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f104d8c-5c79-4fe2-ae67-4349cc28c608 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:06.118913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f104d8c-5c79-4fe2-ae67-4349cc28c608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:06.119204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f104d8c-5c79-4fe2-ae67-4349cc28c608 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:06.119929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f104d8c-5c79-4fe2-ae67-4349cc28c608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:06.121714 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f104d8c-5c79-4fe2-ae67-4349cc28c608 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:06.133392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1250/2478] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:01:06.138554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ca1ae-953c-4b1d-8340-0d2964a52c8b admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:06.149354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ca1ae-953c-4b1d-8340-0d2964a52c8b admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:06.154938 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca28c5-96d9-414b-bda5-ca73f5aa3bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:06.155829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca28c5-96d9-414b-bda5-ca73f5aa3bef admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:06.156111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca28c5-96d9-414b-bda5-ca73f5aa3bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:06.156386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca28c5-96d9-414b-bda5-ca73f5aa3bef admin admin] PATH_INFO: `/v3/projects/2cef7f8b1e6948bcb185caa2d8ec8bb3/users/2333f60db9b2482e9cdc036f242c5485/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:06.158425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca28c5-96d9-414b-bda5-ca73f5aa3bef admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cef7f8b1e6948bcb185caa2d8ec8bb3, user_id=2333f60db9b2482e9cdc036f242c5485, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:06.161118 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca28c5-96d9-414b-bda5-ca73f5aa3bef admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:06.173001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1229/2479] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:01:06 2022] PUT /identity/v3/projects/2cef7f8b1e6948bcb185caa2d8ec8bb3/users/2333f60db9b2482e9cdc036f242c5485/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:06.185077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1c7bb-653f-4ec9-bad6-58ccdf5cd4b5 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:06.185994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1c7bb-653f-4ec9-bad6-58ccdf5cd4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:06.186275 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1c7bb-653f-4ec9-bad6-58ccdf5cd4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:06.214394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-18f1c7bb-653f-4ec9-bad6-58ccdf5cd4b5 None None] MFA Rules not processed for user `2333f60db9b2482e9cdc036f242c5485`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:01:06.215630 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-18f1c7bb-653f-4ec9-bad6-58ccdf5cd4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:06.286868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1251/2480] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:01:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:01:06.297701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca28c5-96d9-414b-bda5-ca73f5aa3bef admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:06.300749 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4fca28c5-96d9-414b-bda5-ca73f5aa3bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:06.358290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca28c5-96d9-414b-bda5-ca73f5aa3bef admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:06.362092 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec04d07-2030-4943-a2ed-35c1cd72fede service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:06.362770 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec04d07-2030-4943-a2ed-35c1cd72fede service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:06.363062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec04d07-2030-4943-a2ed-35c1cd72fede service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:06.363341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec04d07-2030-4943-a2ed-35c1cd72fede service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:06.373449 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec04d07-2030-4943-a2ed-35c1cd72fede service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:06.374818 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec04d07-2030-4943-a2ed-35c1cd72fede service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:06.384920 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1230/2481] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:01:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:01:06.431693 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1c7bb-653f-4ec9-bad6-58ccdf5cd4b5 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:06.433174 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-18f1c7bb-653f-4ec9-bad6-58ccdf5cd4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:06.449590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1c7bb-653f-4ec9-bad6-58ccdf5cd4b5 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:06.453981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe11685-52ef-4e20-9dd5-82aca695998d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:06.454661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe11685-52ef-4e20-9dd5-82aca695998d service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:06.454898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe11685-52ef-4e20-9dd5-82aca695998d service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:06.455184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe11685-52ef-4e20-9dd5-82aca695998d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:06.515130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe11685-52ef-4e20-9dd5-82aca695998d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:06.516657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe11685-52ef-4e20-9dd5-82aca695998d service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:06.526012 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1252/2482] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:01:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:01:12.458254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec04d07-2030-4943-a2ed-35c1cd72fede service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:12.523878 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:12.530372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec04d07-2030-4943-a2ed-35c1cd72fede service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:12.535098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb7ed9-7ac7-471a-ba10-20af9e231163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:12.536006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb7ed9-7ac7-471a-ba10-20af9e231163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:12.536290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb7ed9-7ac7-471a-ba10-20af9e231163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:12.537184 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb7ed9-7ac7-471a-ba10-20af9e231163 admin admin] PATH_INFO: `/v3/users/452b9ca5536d41d192451fb64c9250a4` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:12.537919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb7ed9-7ac7-471a-ba10-20af9e231163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=452b9ca5536d41d192451fb64c9250a4)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:12.539898 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb7ed9-7ac7-471a-ba10-20af9e231163 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:12.734873 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-47cb7ed9-7ac7-471a-ba10-20af9e231163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '452b9ca5536d41d192451fb64c9250a4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:12.817354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-47cb7ed9-7ac7-471a-ba10-20af9e231163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '452b9ca5536d41d192451fb64c9250a4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:12.818029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-47cb7ed9-7ac7-471a-ba10-20af9e231163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '452b9ca5536d41d192451fb64c9250a4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:12.818710 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-47cb7ed9-7ac7-471a-ba10-20af9e231163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '452b9ca5536d41d192451fb64c9250a4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:12.835033 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-47cb7ed9-7ac7-471a-ba10-20af9e231163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '452b9ca5536d41d192451fb64c9250a4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:12.848464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1231/2483] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:01:12 2022] DELETE /identity/v3/users/452b9ca5536d41d192451fb64c9250a4 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:12.853907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477f190-a2fa-4451-9685-b79b829c6404 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:12.854447 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477f190-a2fa-4451-9685-b79b829c6404 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:12.854447 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477f190-a2fa-4451-9685-b79b829c6404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:12.872548 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-d477f190-a2fa-4451-9685-b79b829c6404 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:01:12.873988 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d477f190-a2fa-4451-9685-b79b829c6404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:12.914356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:12.937025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1253/2484] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:01:12.947477 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb7ed9-7ac7-471a-ba10-20af9e231163 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:12.949049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-47cb7ed9-7ac7-471a-ba10-20af9e231163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:13.011421 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb7ed9-7ac7-471a-ba10-20af9e231163 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:13.017200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f3ded-24b5-41c0-aa18-db2e950fbe3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:13.018365 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f3ded-24b5-41c0-aa18-db2e950fbe3a service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:13.018824 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f3ded-24b5-41c0-aa18-db2e950fbe3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:13.019369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f3ded-24b5-41c0-aa18-db2e950fbe3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:13.030099 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f3ded-24b5-41c0-aa18-db2e950fbe3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:13.031561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f3ded-24b5-41c0-aa18-db2e950fbe3a service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:13.042725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1232/2485] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:01:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 11:01:13.516988 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477f190-a2fa-4451-9685-b79b829c6404 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:13.519283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d477f190-a2fa-4451-9685-b79b829c6404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:13.535387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d477f190-a2fa-4451-9685-b79b829c6404 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:13.552807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:13.553782 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:13.554061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:13.554332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74 admin admin] PATH_INFO: `/v3/projects/ed000e041ab74229b0944ef9efa4ddbf` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:13.555559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed000e041ab74229b0944ef9efa4ddbf)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:13.557613 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:13.734112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed000e041ab74229b0944ef9efa4ddbf', 'request_id': 'req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:13.740170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed000e041ab74229b0944ef9efa4ddbf', 'request_id': 'req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:13.749705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed000e041ab74229b0944ef9efa4ddbf', 'request_id': 'req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:13.823582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74 admin admin] The token cache is being invalidate because project ed000e041ab74229b0944ef9efa4ddbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:01:13.823936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:13.824698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:13.827032 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1254/2486] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:01:13 2022] DELETE /identity/v3/projects/ed000e041ab74229b0944ef9efa4ddbf => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:13.831594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f3ded-24b5-41c0-aa18-db2e950fbe3a service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:13.843497 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f3ded-24b5-41c0-aa18-db2e950fbe3a service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:13.902835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d3cc0-3086-4611-81ed-535b8f499fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:13.904516 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d3cc0-3086-4611-81ed-535b8f499fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:13.904800 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d3cc0-3086-4611-81ed-535b8f499fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:13.905204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d3cc0-3086-4611-81ed-535b8f499fa1 admin admin] PATH_INFO: `/v3/users/2333f60db9b2482e9cdc036f242c5485` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:13.906457 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d3cc0-3086-4611-81ed-535b8f499fa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2333f60db9b2482e9cdc036f242c5485)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:13.908856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d3cc0-3086-4611-81ed-535b8f499fa1 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:13.930698 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:14.081458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-023d3cc0-3086-4611-81ed-535b8f499fa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2333f60db9b2482e9cdc036f242c5485'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:14.155017 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-023d3cc0-3086-4611-81ed-535b8f499fa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2333f60db9b2482e9cdc036f242c5485'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:14.155748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-023d3cc0-3086-4611-81ed-535b8f499fa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2333f60db9b2482e9cdc036f242c5485'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:14.156390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-023d3cc0-3086-4611-81ed-535b8f499fa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2333f60db9b2482e9cdc036f242c5485'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:14.170824 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-023d3cc0-3086-4611-81ed-535b8f499fa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2333f60db9b2482e9cdc036f242c5485'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:14.183663 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1233/2487] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:01:13 2022] DELETE /identity/v3/users/2333f60db9b2482e9cdc036f242c5485 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:14.693466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:14.695082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:14.709654 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28dc92-7023-4ee4-9b6d-6e1b8d255f74 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:14.770955 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e899325-50fd-4c6c-95cf-84f0af5cc4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:14.771848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e899325-50fd-4c6c-95cf-84f0af5cc4ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:14.772135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e899325-50fd-4c6c-95cf-84f0af5cc4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:14.772408 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e899325-50fd-4c6c-95cf-84f0af5cc4ae admin admin] PATH_INFO: `/v3/projects/2cef7f8b1e6948bcb185caa2d8ec8bb3` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:14.773624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e899325-50fd-4c6c-95cf-84f0af5cc4ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cef7f8b1e6948bcb185caa2d8ec8bb3)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:14.775573 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e899325-50fd-4c6c-95cf-84f0af5cc4ae admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:14.920780 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3e899325-50fd-4c6c-95cf-84f0af5cc4ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cef7f8b1e6948bcb185caa2d8ec8bb3', 'request_id': 'req-3e899325-50fd-4c6c-95cf-84f0af5cc4ae'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:14.926904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3e899325-50fd-4c6c-95cf-84f0af5cc4ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cef7f8b1e6948bcb185caa2d8ec8bb3', 'request_id': 'req-3e899325-50fd-4c6c-95cf-84f0af5cc4ae'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:14.932644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3e899325-50fd-4c6c-95cf-84f0af5cc4ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cef7f8b1e6948bcb185caa2d8ec8bb3', 'request_id': 'req-3e899325-50fd-4c6c-95cf-84f0af5cc4ae'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:14.941190 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:15.025794 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3e899325-50fd-4c6c-95cf-84f0af5cc4ae admin admin] The token cache is being invalidate because project 2cef7f8b1e6948bcb185caa2d8ec8bb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:01:15.026076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3e899325-50fd-4c6c-95cf-84f0af5cc4ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:15.026728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3e899325-50fd-4c6c-95cf-84f0af5cc4ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:15.028314 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1255/2488] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:01:14 2022] DELETE /identity/v3/projects/2cef7f8b1e6948bcb185caa2d8ec8bb3 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:15.034594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b788d-3827-4bac-9499-9146af986c38 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:15.035110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b788d-3827-4bac-9499-9146af986c38 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:15.035225 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b788d-3827-4bac-9499-9146af986c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:15.053317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-6e2b788d-3827-4bac-9499-9146af986c38 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:01:15.054544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-6e2b788d-3827-4bac-9499-9146af986c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:15.117315 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1234/2489] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:01:15.125914 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e25a8-e1eb-4518-a6c1-52e0546839ba None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:15.126205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e25a8-e1eb-4518-a6c1-52e0546839ba None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:15.126514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e25a8-e1eb-4518-a6c1-52e0546839ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:15.146364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-485e25a8-e1eb-4518-a6c1-52e0546839ba None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:01:15.147573 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-485e25a8-e1eb-4518-a6c1-52e0546839ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:15.156084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1256/2490] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:01:15.160624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b788d-3827-4bac-9499-9146af986c38 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:15.171702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b788d-3827-4bac-9499-9146af986c38 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:15.177487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b484b7-b00d-46dd-986e-28941f6c90cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:15.178156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b484b7-b00d-46dd-986e-28941f6c90cd admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:15.178437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b484b7-b00d-46dd-986e-28941f6c90cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:15.178710 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b484b7-b00d-46dd-986e-28941f6c90cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:15.179451 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b484b7-b00d-46dd-986e-28941f6c90cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:01:15.179758 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b484b7-b00d-46dd-986e-28941f6c90cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:15.181520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b484b7-b00d-46dd-986e-28941f6c90cd admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:15.194940 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1235/2491] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:01:15.199163 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e25a8-e1eb-4518-a6c1-52e0546839ba None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:15.210304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e25a8-e1eb-4518-a6c1-52e0546839ba None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:15.215031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6379e6-53a8-4d66-a49e-ef57d0fa89d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:15.215676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6379e6-53a8-4d66-a49e-ef57d0fa89d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:15.216274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6379e6-53a8-4d66-a49e-ef57d0fa89d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:15.216901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6379e6-53a8-4d66-a49e-ef57d0fa89d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:15.217649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6379e6-53a8-4d66-a49e-ef57d0fa89d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:15.219560 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6379e6-53a8-4d66-a49e-ef57d0fa89d4 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:15.293791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1257/2492] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:01:15 2022] POST /identity/v3/projects => generated 369 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:01:15.298153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b484b7-b00d-46dd-986e-28941f6c90cd admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:15.308175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b484b7-b00d-46dd-986e-28941f6c90cd admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:15.368651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b189352-6942-43a8-9c79-8eac1bc23a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:15.369372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b189352-6942-43a8-9c79-8eac1bc23a77 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:15.369657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b189352-6942-43a8-9c79-8eac1bc23a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:15.369955 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b189352-6942-43a8-9c79-8eac1bc23a77 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:15.370638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b189352-6942-43a8-9c79-8eac1bc23a77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:15.373458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b189352-6942-43a8-9c79-8eac1bc23a77 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:15.490719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1236/2493] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:01:15 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:01:15.495151 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6379e6-53a8-4d66-a49e-ef57d0fa89d4 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:15.507077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6379e6-53a8-4d66-a49e-ef57d0fa89d4 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:15.512130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49879a3a-db67-4de0-a633-909fbf3f5410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:15.512587 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49879a3a-db67-4de0-a633-909fbf3f5410 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:15.513089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49879a3a-db67-4de0-a633-909fbf3f5410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:15.513844 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49879a3a-db67-4de0-a633-909fbf3f5410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:15.514612 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49879a3a-db67-4de0-a633-909fbf3f5410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:15.516235 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49879a3a-db67-4de0-a633-909fbf3f5410 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:15.529727 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1258/2494] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:01:15.533720 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b189352-6942-43a8-9c79-8eac1bc23a77 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:15.545165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b189352-6942-43a8-9c79-8eac1bc23a77 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:15.550089 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdd4d7-a995-4977-9d3a-053a8cdf194d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:15.550757 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdd4d7-a995-4977-9d3a-053a8cdf194d admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:15.551040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdd4d7-a995-4977-9d3a-053a8cdf194d admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:15.551317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdd4d7-a995-4977-9d3a-053a8cdf194d admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:15.552048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fdd4d7-a995-4977-9d3a-053a8cdf194d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:15.553685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fdd4d7-a995-4977-9d3a-053a8cdf194d admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:15.564697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1237/2495] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:01:15.568920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49879a3a-db67-4de0-a633-909fbf3f5410 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:15.579959 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49879a3a-db67-4de0-a633-909fbf3f5410 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:15.584402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56a18d-e8c9-4177-867c-6ee0c2949186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:15.585310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56a18d-e8c9-4177-867c-6ee0c2949186 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:15.585598 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56a18d-e8c9-4177-867c-6ee0c2949186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:15.585875 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56a18d-e8c9-4177-867c-6ee0c2949186 admin admin] PATH_INFO: `/v3/projects/a44bfda344c140e1897ba0054af99e71/users/96c442744a5743b381f64a5d0b5d4b79/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:15.600842 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56a18d-e8c9-4177-867c-6ee0c2949186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a44bfda344c140e1897ba0054af99e71, user_id=96c442744a5743b381f64a5d0b5d4b79, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:15.603941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56a18d-e8c9-4177-867c-6ee0c2949186 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:15.686146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1259/2496] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:01:15 2022] PUT /identity/v3/projects/a44bfda344c140e1897ba0054af99e71/users/96c442744a5743b381f64a5d0b5d4b79/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:15.695962 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e1969-8b34-47bf-a0d8-ab940acb583d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:15.701404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e1969-8b34-47bf-a0d8-ab940acb583d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:15.701404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e1969-8b34-47bf-a0d8-ab940acb583d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:15.704271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:15.726340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-681e1969-8b34-47bf-a0d8-ab940acb583d None None] MFA Rules not processed for user `96c442744a5743b381f64a5d0b5d4b79`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:01:15.729257 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-681e1969-8b34-47bf-a0d8-ab940acb583d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:15.797737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1238/2497] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:01:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:01:15.808606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56a18d-e8c9-4177-867c-6ee0c2949186 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:15.810689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2a56a18d-e8c9-4177-867c-6ee0c2949186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:15.874613 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56a18d-e8c9-4177-867c-6ee0c2949186 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:15.879225 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1997441b-1211-4739-ba4a-54a261dca0f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:15.879954 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997441b-1211-4739-ba4a-54a261dca0f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:15.880339 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997441b-1211-4739-ba4a-54a261dca0f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:15.880712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997441b-1211-4739-ba4a-54a261dca0f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:15.891841 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997441b-1211-4739-ba4a-54a261dca0f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:15.893318 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1997441b-1211-4739-ba4a-54a261dca0f9 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:15.902523 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1260/2498] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:01:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:01:15.922585 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e1969-8b34-47bf-a0d8-ab940acb583d None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:15.987164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e1969-8b34-47bf-a0d8-ab940acb583d None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:15.991780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c16d3-c9bc-4456-9613-ff27d300e49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:15.992652 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c16d3-c9bc-4456-9613-ff27d300e49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:15.992930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c16d3-c9bc-4456-9613-ff27d300e49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:15.993237 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c16d3-c9bc-4456-9613-ff27d300e49a admin admin] PATH_INFO: `/v3/users/96c442744a5743b381f64a5d0b5d4b79` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:15.994422 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c16d3-c9bc-4456-9613-ff27d300e49a admin admin] RBAC: Authorizing `identity:delete_user(user_id=96c442744a5743b381f64a5d0b5d4b79)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:15.996321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c16d3-c9bc-4456-9613-ff27d300e49a admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:16.199103 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-067c16d3-c9bc-4456-9613-ff27d300e49a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96c442744a5743b381f64a5d0b5d4b79'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:16.274405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-067c16d3-c9bc-4456-9613-ff27d300e49a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96c442744a5743b381f64a5d0b5d4b79'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:16.275109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-067c16d3-c9bc-4456-9613-ff27d300e49a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96c442744a5743b381f64a5d0b5d4b79'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:16.275748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-067c16d3-c9bc-4456-9613-ff27d300e49a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96c442744a5743b381f64a5d0b5d4b79'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:16.288986 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-067c16d3-c9bc-4456-9613-ff27d300e49a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96c442744a5743b381f64a5d0b5d4b79'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:16.301854 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1239/2499] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:01:15 2022] DELETE /identity/v3/users/96c442744a5743b381f64a5d0b5d4b79 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:16.306379 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221caab-1b36-4c73-be53-8f8e99881feb None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:16.306750 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221caab-1b36-4c73-be53-8f8e99881feb None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:16.307029 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221caab-1b36-4c73-be53-8f8e99881feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:16.324055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-4221caab-1b36-4c73-be53-8f8e99881feb None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:01:16.325810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-4221caab-1b36-4c73-be53-8f8e99881feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:16.386010 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1261/2500] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:01:16.396606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c16d3-c9bc-4456-9613-ff27d300e49a admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:16.398063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-067c16d3-c9bc-4456-9613-ff27d300e49a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:16.413406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:16.458290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c16d3-c9bc-4456-9613-ff27d300e49a admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:16.462069 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc33a5-b6fc-4986-841b-b7314d9146ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:16.465388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc33a5-b6fc-4986-841b-b7314d9146ae service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:16.465388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc33a5-b6fc-4986-841b-b7314d9146ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:16.465388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc33a5-b6fc-4986-841b-b7314d9146ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:16.474065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc33a5-b6fc-4986-841b-b7314d9146ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:16.475504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc33a5-b6fc-4986-841b-b7314d9146ae service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:16.486174 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1240/2501] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:01:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:01:16.928283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221caab-1b36-4c73-be53-8f8e99881feb None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:16.930086 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-4221caab-1b36-4c73-be53-8f8e99881feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:16.949657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221caab-1b36-4c73-be53-8f8e99881feb None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:16.957956 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af40f9-f962-4bf0-b1aa-089fe8510f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:16.958947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af40f9-f962-4bf0-b1aa-089fe8510f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:16.959316 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af40f9-f962-4bf0-b1aa-089fe8510f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:16.959700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af40f9-f962-4bf0-b1aa-089fe8510f2e admin admin] PATH_INFO: `/v3/projects/a44bfda344c140e1897ba0054af99e71` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:16.961331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af40f9-f962-4bf0-b1aa-089fe8510f2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a44bfda344c140e1897ba0054af99e71)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:16.963726 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af40f9-f962-4bf0-b1aa-089fe8510f2e admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:17.051302 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:17.064721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-b9af40f9-f962-4bf0-b1aa-089fe8510f2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a44bfda344c140e1897ba0054af99e71', 'request_id': 'req-b9af40f9-f962-4bf0-b1aa-089fe8510f2e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:17.071144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-b9af40f9-f962-4bf0-b1aa-089fe8510f2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a44bfda344c140e1897ba0054af99e71', 'request_id': 'req-b9af40f9-f962-4bf0-b1aa-089fe8510f2e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:17.077247 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-b9af40f9-f962-4bf0-b1aa-089fe8510f2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a44bfda344c140e1897ba0054af99e71', 'request_id': 'req-b9af40f9-f962-4bf0-b1aa-089fe8510f2e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:17.130873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-b9af40f9-f962-4bf0-b1aa-089fe8510f2e admin admin] The token cache is being invalidate because project a44bfda344c140e1897ba0054af99e71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:01:17.131161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-b9af40f9-f962-4bf0-b1aa-089fe8510f2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:17.131803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-b9af40f9-f962-4bf0-b1aa-089fe8510f2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:17.133501 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1262/2502] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:01:16 2022] DELETE /identity/v3/projects/a44bfda344c140e1897ba0054af99e71 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:17.139437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f0409-c2b5-46b5-be31-76b9ff4b3d33 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:17.139729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f0409-c2b5-46b5-be31-76b9ff4b3d33 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:17.140036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f0409-c2b5-46b5-be31-76b9ff4b3d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:17.157689 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a65f0409-c2b5-46b5-be31-76b9ff4b3d33 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:01:17.158966 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a65f0409-c2b5-46b5-be31-76b9ff4b3d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:17.221117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1241/2503] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:01:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:01:17.230310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d7a74-fc00-48e7-981d-40b62d201761 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:17.230599 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d7a74-fc00-48e7-981d-40b62d201761 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:17.230876 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d7a74-fc00-48e7-981d-40b62d201761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:17.250064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-5f5d7a74-fc00-48e7-981d-40b62d201761 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:01:17.251365 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5f5d7a74-fc00-48e7-981d-40b62d201761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:17.259394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1263/2504] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:01:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:01:17.263689 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f0409-c2b5-46b5-be31-76b9ff4b3d33 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:17.273682 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f0409-c2b5-46b5-be31-76b9ff4b3d33 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:17.278239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974c961-dfa4-4676-bdde-f6203ba680a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:17.279416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974c961-dfa4-4676-bdde-f6203ba680a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:17.279702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974c961-dfa4-4676-bdde-f6203ba680a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:17.279980 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974c961-dfa4-4676-bdde-f6203ba680a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:17.280713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974c961-dfa4-4676-bdde-f6203ba680a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:01:17.281052 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974c961-dfa4-4676-bdde-f6203ba680a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:17.282689 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974c961-dfa4-4676-bdde-f6203ba680a1 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:17.294298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1242/2505] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:01:17.300209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d7a74-fc00-48e7-981d-40b62d201761 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:17.309478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d7a74-fc00-48e7-981d-40b62d201761 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:17.314888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b19f3a-e75a-481a-b1f5-cc74d28c3d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:17.315531 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b19f3a-e75a-481a-b1f5-cc74d28c3d13 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:17.315813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b19f3a-e75a-481a-b1f5-cc74d28c3d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:17.316083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b19f3a-e75a-481a-b1f5-cc74d28c3d13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:17.316764 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b19f3a-e75a-481a-b1f5-cc74d28c3d13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:17.318703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b19f3a-e75a-481a-b1f5-cc74d28c3d13 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:17.401742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1264/2506] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:01:17 2022] POST /identity/v3/projects => generated 361 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:01:17.406655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974c961-dfa4-4676-bdde-f6203ba680a1 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:17.416823 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974c961-dfa4-4676-bdde-f6203ba680a1 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:17.485582 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc18ce0-cb31-4c22-9c2b-942b687bf000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:17.486182 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc18ce0-cb31-4c22-9c2b-942b687bf000 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:17.486464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc18ce0-cb31-4c22-9c2b-942b687bf000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:17.486738 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc18ce0-cb31-4c22-9c2b-942b687bf000 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:17.487446 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc18ce0-cb31-4c22-9c2b-942b687bf000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:17.489512 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc18ce0-cb31-4c22-9c2b-942b687bf000 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:17.578239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1243/2507] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:01:17 2022] POST /identity/v3/users => generated 281 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:01:17.582193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b19f3a-e75a-481a-b1f5-cc74d28c3d13 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:17.593271 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b19f3a-e75a-481a-b1f5-cc74d28c3d13 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:17.598224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a5917-9324-4cff-955c-b73142b6e0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:17.598881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a5917-9324-4cff-955c-b73142b6e0a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:17.599162 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a5917-9324-4cff-955c-b73142b6e0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:17.599438 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a5917-9324-4cff-955c-b73142b6e0a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:17.600168 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a5917-9324-4cff-955c-b73142b6e0a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:17.601796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a5917-9324-4cff-955c-b73142b6e0a6 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:17.612872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1265/2508] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:01:17.616936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc18ce0-cb31-4c22-9c2b-942b687bf000 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:17.627758 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc18ce0-cb31-4c22-9c2b-942b687bf000 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:17.632407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b5aa5-a8b6-4d22-bed3-ba74d930bf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:17.633085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b5aa5-a8b6-4d22-bed3-ba74d930bf2c admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:17.633376 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b5aa5-a8b6-4d22-bed3-ba74d930bf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:17.633654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b5aa5-a8b6-4d22-bed3-ba74d930bf2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:17.634384 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b5aa5-a8b6-4d22-bed3-ba74d930bf2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:17.635963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b5aa5-a8b6-4d22-bed3-ba74d930bf2c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:17.647175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1244/2509] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:01:17.651256 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a5917-9324-4cff-955c-b73142b6e0a6 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:17.662226 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a5917-9324-4cff-955c-b73142b6e0a6 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:17.666823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ff6bb-292d-4b66-afef-aa0875e0b225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:17.667698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ff6bb-292d-4b66-afef-aa0875e0b225 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:17.667980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ff6bb-292d-4b66-afef-aa0875e0b225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:17.668260 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ff6bb-292d-4b66-afef-aa0875e0b225 admin admin] PATH_INFO: `/v3/projects/b2b707ec17f94a0089f354ecfd7a962a/users/2cbf2364a2014fd68bfa44df5c8a9300/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:17.683665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ff6bb-292d-4b66-afef-aa0875e0b225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b707ec17f94a0089f354ecfd7a962a, user_id=2cbf2364a2014fd68bfa44df5c8a9300, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:17.686589 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ff6bb-292d-4b66-afef-aa0875e0b225 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:17.771431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1266/2510] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:01:17 2022] PUT /identity/v3/projects/b2b707ec17f94a0089f354ecfd7a962a/users/2cbf2364a2014fd68bfa44df5c8a9300/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:17.782592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44703f49-a163-4a98-80b3-9b922aefeeff None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:17.782979 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44703f49-a163-4a98-80b3-9b922aefeeff None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:17.783162 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44703f49-a163-4a98-80b3-9b922aefeeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:17.811176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-44703f49-a163-4a98-80b3-9b922aefeeff None None] MFA Rules not processed for user `2cbf2364a2014fd68bfa44df5c8a9300`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:01:17.812448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-44703f49-a163-4a98-80b3-9b922aefeeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:17.821835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:17.881515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1245/2511] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:01:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:01:17.913795 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ff6bb-292d-4b66-afef-aa0875e0b225 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:17.915511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-647ff6bb-292d-4b66-afef-aa0875e0b225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:17.973872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ff6bb-292d-4b66-afef-aa0875e0b225 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:17.977980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c33476-21e7-4512-bb58-38e99e43d83b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:17.978664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c33476-21e7-4512-bb58-38e99e43d83b service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:17.978953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c33476-21e7-4512-bb58-38e99e43d83b service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:17.979234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c33476-21e7-4512-bb58-38e99e43d83b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:17.988988 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c33476-21e7-4512-bb58-38e99e43d83b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:17.990448 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c33476-21e7-4512-bb58-38e99e43d83b service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:17.999910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1267/2512] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:01:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:01:18.014823 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44703f49-a163-4a98-80b3-9b922aefeeff None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:18.076591 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44703f49-a163-4a98-80b3-9b922aefeeff None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:18.081277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ee8af-312a-4a09-abcb-ad47441f68bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:18.082145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ee8af-312a-4a09-abcb-ad47441f68bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:18.082426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ee8af-312a-4a09-abcb-ad47441f68bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:18.082699 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ee8af-312a-4a09-abcb-ad47441f68bb admin admin] PATH_INFO: `/v3/users/2cbf2364a2014fd68bfa44df5c8a9300` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:18.083908 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ee8af-312a-4a09-abcb-ad47441f68bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cbf2364a2014fd68bfa44df5c8a9300)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:18.085852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ee8af-312a-4a09-abcb-ad47441f68bb admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:18.306760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e85ee8af-312a-4a09-abcb-ad47441f68bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cbf2364a2014fd68bfa44df5c8a9300'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:18.403875 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e85ee8af-312a-4a09-abcb-ad47441f68bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cbf2364a2014fd68bfa44df5c8a9300'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:18.405244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e85ee8af-312a-4a09-abcb-ad47441f68bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cbf2364a2014fd68bfa44df5c8a9300'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:18.405870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e85ee8af-312a-4a09-abcb-ad47441f68bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cbf2364a2014fd68bfa44df5c8a9300'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:18.420164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e85ee8af-312a-4a09-abcb-ad47441f68bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cbf2364a2014fd68bfa44df5c8a9300'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:18.432600 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1246/2513] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:01:18 2022] DELETE /identity/v3/users/2cbf2364a2014fd68bfa44df5c8a9300 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:18.437257 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea2038-72fc-40bc-b047-08ed73c14336 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:18.437554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea2038-72fc-40bc-b047-08ed73c14336 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:18.437836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea2038-72fc-40bc-b047-08ed73c14336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:18.455671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-7eea2038-72fc-40bc-b047-08ed73c14336 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:01:18.459724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7eea2038-72fc-40bc-b047-08ed73c14336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:18.508724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:18.518748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1268/2514] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:01:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:01:18.531161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ee8af-312a-4a09-abcb-ad47441f68bb admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:18.532630 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e85ee8af-312a-4a09-abcb-ad47441f68bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:18.565928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:18.591341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ee8af-312a-4a09-abcb-ad47441f68bb admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:18.595745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cb25f-ae87-4fa6-900c-397b4b6bf6df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:18.597402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cb25f-ae87-4fa6-900c-397b4b6bf6df service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:18.597402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cb25f-ae87-4fa6-900c-397b4b6bf6df service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:18.597402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cb25f-ae87-4fa6-900c-397b4b6bf6df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:18.606371 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cb25f-ae87-4fa6-900c-397b4b6bf6df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:18.608209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cb25f-ae87-4fa6-900c-397b4b6bf6df service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:18.617739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1247/2515] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:01:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 11:01:19.036291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea2038-72fc-40bc-b047-08ed73c14336 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:19.038323 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7eea2038-72fc-40bc-b047-08ed73c14336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:19.051604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea2038-72fc-40bc-b047-08ed73c14336 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:19.056837 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe13387-6fe2-4696-b04f-205784c77aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:19.057868 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe13387-6fe2-4696-b04f-205784c77aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:19.058154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe13387-6fe2-4696-b04f-205784c77aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:19.058426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe13387-6fe2-4696-b04f-205784c77aec admin admin] PATH_INFO: `/v3/projects/b2b707ec17f94a0089f354ecfd7a962a` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:19.059687 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe13387-6fe2-4696-b04f-205784c77aec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2b707ec17f94a0089f354ecfd7a962a)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:19.068025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe13387-6fe2-4696-b04f-205784c77aec admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:19.214303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0fe13387-6fe2-4696-b04f-205784c77aec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2b707ec17f94a0089f354ecfd7a962a', 'request_id': 'req-0fe13387-6fe2-4696-b04f-205784c77aec'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:19.221343 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0fe13387-6fe2-4696-b04f-205784c77aec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2b707ec17f94a0089f354ecfd7a962a', 'request_id': 'req-0fe13387-6fe2-4696-b04f-205784c77aec'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:19.227032 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0fe13387-6fe2-4696-b04f-205784c77aec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2b707ec17f94a0089f354ecfd7a962a', 'request_id': 'req-0fe13387-6fe2-4696-b04f-205784c77aec'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:19.306938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0fe13387-6fe2-4696-b04f-205784c77aec admin admin] The token cache is being invalidate because project b2b707ec17f94a0089f354ecfd7a962a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:01:19.307577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0fe13387-6fe2-4696-b04f-205784c77aec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:19.308405 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0fe13387-6fe2-4696-b04f-205784c77aec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:01:19.310816 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1269/2516] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:01:19 2022] DELETE /identity/v3/projects/b2b707ec17f94a0089f354ecfd7a962a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:01:28.897290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=13 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:29.303520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:29.802076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cb25f-ae87-4fa6-900c-397b4b6bf6df service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:29.803473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-297cb25f-ae87-4fa6-900c-397b4b6bf6df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:29.889609 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cb25f-ae87-4fa6-900c-397b4b6bf6df service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:29.951205 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc33819-9191-44e6-a604-308e6f86603b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:29.951888 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc33819-9191-44e6-a604-308e6f86603b service placement] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:29.952075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc33819-9191-44e6-a604-308e6f86603b service placement] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:29.952353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc33819-9191-44e6-a604-308e6f86603b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:29.954964 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-5cc33819-9191-44e6-a604-308e6f86603b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:29.966411 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc33819-9191-44e6-a604-308e6f86603b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:29.968044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc33819-9191-44e6-a604-308e6f86603b service placement] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:29.978288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1248/2517] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:01:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:01:33.015639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe13387-6fe2-4696-b04f-205784c77aec admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:01:33.017816 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0fe13387-6fe2-4696-b04f-205784c77aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:33.097916 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe13387-6fe2-4696-b04f-205784c77aec admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:01:33.103416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26351433-5aff-44d5-8381-a8415bdcba35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:01:33.104112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26351433-5aff-44d5-8381-a8415bdcba35 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:01:33.104393 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26351433-5aff-44d5-8381-a8415bdcba35 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:01:33.104669 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26351433-5aff-44d5-8381-a8415bdcba35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:01:33.106778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-26351433-5aff-44d5-8381-a8415bdcba35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:01:33.179317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26351433-5aff-44d5-8381-a8415bdcba35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:01:33.180913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26351433-5aff-44d5-8381-a8415bdcba35 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:01:33.190701 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1270/2518] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:01:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:01:39.974060 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=63 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:01:43.185897 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=61 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:10.129704 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc33819-9191-44e6-a604-308e6f86603b service placement] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:10.131114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-5cc33819-9191-44e6-a604-308e6f86603b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:10.191089 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc33819-9191-44e6-a604-308e6f86603b service placement] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:10.194730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb85dd-3620-4d99-aeac-ea8b597c3b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:10.195379 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb85dd-3620-4d99-aeac-ea8b597c3b79 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:10.195663 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb85dd-3620-4d99-aeac-ea8b597c3b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:10.195946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb85dd-3620-4d99-aeac-ea8b597c3b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:10.197955 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d0bb85dd-3620-4d99-aeac-ea8b597c3b79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:10.284438 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb85dd-3620-4d99-aeac-ea8b597c3b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:10.286370 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb85dd-3620-4d99-aeac-ea8b597c3b79 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:10.306433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1249/2519] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 11:02:12.079974 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26351433-5aff-44d5-8381-a8415bdcba35 service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:12.082172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-26351433-5aff-44d5-8381-a8415bdcba35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:12.096570 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26351433-5aff-44d5-8381-a8415bdcba35 service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:12.100745 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d05b27-b6bd-4823-8bea-e91ac1af8f24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:12.101446 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d05b27-b6bd-4823-8bea-e91ac1af8f24 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:12.101731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d05b27-b6bd-4823-8bea-e91ac1af8f24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:12.102015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d05b27-b6bd-4823-8bea-e91ac1af8f24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:12.104476 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-34d05b27-b6bd-4823-8bea-e91ac1af8f24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:12.118522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d05b27-b6bd-4823-8bea-e91ac1af8f24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:12.119986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d05b27-b6bd-4823-8bea-e91ac1af8f24 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:12.134037 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1271/2520] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:14.135581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb85dd-3620-4d99-aeac-ea8b597c3b79 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:14.137174 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d0bb85dd-3620-4d99-aeac-ea8b597c3b79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:14.153179 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb85dd-3620-4d99-aeac-ea8b597c3b79 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:14.158650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea540e65-0eb5-4464-8d9b-dfb29718bb5e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:14.159095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea540e65-0eb5-4464-8d9b-dfb29718bb5e service placement] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:14.159385 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea540e65-0eb5-4464-8d9b-dfb29718bb5e service placement] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:14.159664 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea540e65-0eb5-4464-8d9b-dfb29718bb5e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:14.161607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ea540e65-0eb5-4464-8d9b-dfb29718bb5e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:14.175657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea540e65-0eb5-4464-8d9b-dfb29718bb5e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:14.177720 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea540e65-0eb5-4464-8d9b-dfb29718bb5e service placement] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:14.189819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1250/2521] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 11:02:14.286448 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d05b27-b6bd-4823-8bea-e91ac1af8f24 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:14.288108 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-34d05b27-b6bd-4823-8bea-e91ac1af8f24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:14.300874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d05b27-b6bd-4823-8bea-e91ac1af8f24 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:14.326913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca8e4a-8594-4adb-b4b9-203f89f671d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:14.326913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca8e4a-8594-4adb-b4b9-203f89f671d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:14.326913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca8e4a-8594-4adb-b4b9-203f89f671d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:14.326913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca8e4a-8594-4adb-b4b9-203f89f671d8 admin admin] PATH_INFO: `/v3/users/bbb69faf3ec54489a92fe87935f2f777` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:14.328313 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca8e4a-8594-4adb-b4b9-203f89f671d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbb69faf3ec54489a92fe87935f2f777)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:14.328313 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca8e4a-8594-4adb-b4b9-203f89f671d8 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:14.526037 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a9ca8e4a-8594-4adb-b4b9-203f89f671d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbb69faf3ec54489a92fe87935f2f777'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:14.549132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a9ca8e4a-8594-4adb-b4b9-203f89f671d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbb69faf3ec54489a92fe87935f2f777'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:14.647362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a9ca8e4a-8594-4adb-b4b9-203f89f671d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbb69faf3ec54489a92fe87935f2f777'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:14.657047 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a9ca8e4a-8594-4adb-b4b9-203f89f671d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbb69faf3ec54489a92fe87935f2f777'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:14.657917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a9ca8e4a-8594-4adb-b4b9-203f89f671d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbb69faf3ec54489a92fe87935f2f777'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:14.668024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1272/2522] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:02:14 2022] DELETE /identity/v3/users/bbb69faf3ec54489a92fe87935f2f777 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:14.939509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea540e65-0eb5-4464-8d9b-dfb29718bb5e service placement] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:14.955849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea540e65-0eb5-4464-8d9b-dfb29718bb5e service placement] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:15.033547 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1aea2-4756-4ec5-8c42-f6e5b9af02f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:15.034433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1aea2-4756-4ec5-8c42-f6e5b9af02f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:15.034722 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1aea2-4756-4ec5-8c42-f6e5b9af02f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:15.035005 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1aea2-4756-4ec5-8c42-f6e5b9af02f2 admin admin] PATH_INFO: `/v3/projects/bbcfca56769742b9b3645f8f7c9bc6e3` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:15.036233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1aea2-4756-4ec5-8c42-f6e5b9af02f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbcfca56769742b9b3645f8f7c9bc6e3)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:15.338566 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1aea2-4756-4ec5-8c42-f6e5b9af02f2 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:15.338566 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:15.338566 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-09c1aea2-4756-4ec5-8c42-f6e5b9af02f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbcfca56769742b9b3645f8f7c9bc6e3', 'request_id': 'req-09c1aea2-4756-4ec5-8c42-f6e5b9af02f2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:15.338566 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-09c1aea2-4756-4ec5-8c42-f6e5b9af02f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbcfca56769742b9b3645f8f7c9bc6e3', 'request_id': 'req-09c1aea2-4756-4ec5-8c42-f6e5b9af02f2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:15.338566 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-09c1aea2-4756-4ec5-8c42-f6e5b9af02f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbcfca56769742b9b3645f8f7c9bc6e3', 'request_id': 'req-09c1aea2-4756-4ec5-8c42-f6e5b9af02f2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:15.344043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-09c1aea2-4756-4ec5-8c42-f6e5b9af02f2 admin admin] The token cache is being invalidate because project bbcfca56769742b9b3645f8f7c9bc6e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:02:15.344531 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-09c1aea2-4756-4ec5-8c42-f6e5b9af02f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:15.345835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-09c1aea2-4756-4ec5-8c42-f6e5b9af02f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:15.347593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1251/2523] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:02:14 2022] DELETE /identity/v3/projects/bbcfca56769742b9b3645f8f7c9bc6e3 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:15.353387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d228ff-e1f1-45bc-9347-2d301b61b8a0 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:15.353966 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d228ff-e1f1-45bc-9347-2d301b61b8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:15.353966 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d228ff-e1f1-45bc-9347-2d301b61b8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:15.390869 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-03d228ff-e1f1-45bc-9347-2d301b61b8a0 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:15.392213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-03d228ff-e1f1-45bc-9347-2d301b61b8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:15.462892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1273/2524] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:15.471931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257a70f-c64d-471a-b947-c611be8365ce None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:15.472332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257a70f-c64d-471a-b947-c611be8365ce None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:15.472807 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257a70f-c64d-471a-b947-c611be8365ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:15.501473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-5257a70f-c64d-471a-b947-c611be8365ce None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:15.502928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-5257a70f-c64d-471a-b947-c611be8365ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:15.511562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1252/2525] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:15.516160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d228ff-e1f1-45bc-9347-2d301b61b8a0 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:15.529168 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d228ff-e1f1-45bc-9347-2d301b61b8a0 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:15.535524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c2973-d760-410d-87d7-3ad7a9b8861d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:15.536643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c2973-d760-410d-87d7-3ad7a9b8861d admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:15.539241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c2973-d760-410d-87d7-3ad7a9b8861d admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:15.539241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c2973-d760-410d-87d7-3ad7a9b8861d admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:15.539241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c2973-d760-410d-87d7-3ad7a9b8861d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:02:15.539241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c2973-d760-410d-87d7-3ad7a9b8861d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:15.541621 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c2973-d760-410d-87d7-3ad7a9b8861d admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:15.558020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1274/2526] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:02:15.562503 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257a70f-c64d-471a-b947-c611be8365ce None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:15.573399 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257a70f-c64d-471a-b947-c611be8365ce None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:15.578265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919320b-c09d-4503-b639-83411c457e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:15.578936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919320b-c09d-4503-b639-83411c457e9a admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:15.579220 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919320b-c09d-4503-b639-83411c457e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:15.579500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919320b-c09d-4503-b639-83411c457e9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:15.580199 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919320b-c09d-4503-b639-83411c457e9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:15.582159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919320b-c09d-4503-b639-83411c457e9a admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:15.688779 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1253/2527] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:02:15 2022] POST /identity/v3/projects => generated 369 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:02:15.693087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c2973-d760-410d-87d7-3ad7a9b8861d admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:15.704816 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c2973-d760-410d-87d7-3ad7a9b8861d admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:15.764250 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:15.825337 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c53b9-1119-49bd-bac2-731be72f5d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:15.826539 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c53b9-1119-49bd-bac2-731be72f5d4a admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:15.827054 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c53b9-1119-49bd-bac2-731be72f5d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:15.827610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c53b9-1119-49bd-bac2-731be72f5d4a admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:15.839959 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c53b9-1119-49bd-bac2-731be72f5d4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:15.847415 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c53b9-1119-49bd-bac2-731be72f5d4a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:15.971467 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1275/2528] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:02:15 2022] POST /identity/v3/users => generated 285 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:02:15.975331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919320b-c09d-4503-b639-83411c457e9a admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:15.986632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919320b-c09d-4503-b639-83411c457e9a admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:15.991430 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6b2fe-baab-4d5a-ae9d-89e8984d6818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:15.992114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6b2fe-baab-4d5a-ae9d-89e8984d6818 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:15.992500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6b2fe-baab-4d5a-ae9d-89e8984d6818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:15.993529 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6b2fe-baab-4d5a-ae9d-89e8984d6818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:15.994273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6b2fe-baab-4d5a-ae9d-89e8984d6818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:15.996071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6b2fe-baab-4d5a-ae9d-89e8984d6818 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:16.014970 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1254/2529] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:16.015814 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c53b9-1119-49bd-bac2-731be72f5d4a admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:16.029120 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c53b9-1119-49bd-bac2-731be72f5d4a admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:16.034644 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c59b6-4dbb-44f8-98ed-33d154b31679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:16.035835 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c59b6-4dbb-44f8-98ed-33d154b31679 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:16.036197 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c59b6-4dbb-44f8-98ed-33d154b31679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:16.036586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c59b6-4dbb-44f8-98ed-33d154b31679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:16.037439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c59b6-4dbb-44f8-98ed-33d154b31679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:16.039246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c59b6-4dbb-44f8-98ed-33d154b31679 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:16.056908 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1276/2530] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:16.060730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6b2fe-baab-4d5a-ae9d-89e8984d6818 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:16.072553 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6b2fe-baab-4d5a-ae9d-89e8984d6818 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:16.077976 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d453a21-5668-4c66-a138-64a0761d7fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:16.078996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d453a21-5668-4c66-a138-64a0761d7fac admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:16.081602 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d453a21-5668-4c66-a138-64a0761d7fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:16.081999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d453a21-5668-4c66-a138-64a0761d7fac admin admin] PATH_INFO: `/v3/projects/f01e7ea926544200a89bc6554fe7ad11/users/68e854ada59343f2bea86961e83eda4e/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:16.101135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d453a21-5668-4c66-a138-64a0761d7fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f01e7ea926544200a89bc6554fe7ad11, user_id=68e854ada59343f2bea86961e83eda4e, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:16.105995 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d453a21-5668-4c66-a138-64a0761d7fac admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:16.191251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1255/2531] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:16 2022] PUT /identity/v3/projects/f01e7ea926544200a89bc6554fe7ad11/users/68e854ada59343f2bea86961e83eda4e/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:16.193022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7536734-0736-4b15-88c3-8b1b3d52763c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:16.194839 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7536734-0736-4b15-88c3-8b1b3d52763c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:16.194839 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7536734-0736-4b15-88c3-8b1b3d52763c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:16.230041 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-f7536734-0736-4b15-88c3-8b1b3d52763c None None] MFA Rules not processed for user `68e854ada59343f2bea86961e83eda4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:16.231554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f7536734-0736-4b15-88c3-8b1b3d52763c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:16.332084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1277/2532] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:16 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:16.343662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d453a21-5668-4c66-a138-64a0761d7fac admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:16.345965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1d453a21-5668-4c66-a138-64a0761d7fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:16.439079 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d453a21-5668-4c66-a138-64a0761d7fac admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:16.449004 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467842c-4462-45ae-b945-19200b6344ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:16.449540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467842c-4462-45ae-b945-19200b6344ca service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:16.450020 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467842c-4462-45ae-b945-19200b6344ca service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:16.450522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467842c-4462-45ae-b945-19200b6344ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:16.464739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467842c-4462-45ae-b945-19200b6344ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:16.467679 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467842c-4462-45ae-b945-19200b6344ca service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:16.483881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1256/2533] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:16 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:16.797689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7536734-0736-4b15-88c3-8b1b3d52763c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:16.799767 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f7536734-0736-4b15-88c3-8b1b3d52763c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:16.874555 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7536734-0736-4b15-88c3-8b1b3d52763c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:16.879383 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59463ae3-8073-4b2d-ad87-fdb4486af77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:16.880217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59463ae3-8073-4b2d-ad87-fdb4486af77a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:16.880511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59463ae3-8073-4b2d-ad87-fdb4486af77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:16.880797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59463ae3-8073-4b2d-ad87-fdb4486af77a admin admin] PATH_INFO: `/v3/users/9f1e192f3eab42ee9f9ed5d2789e0d1e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:16.882198 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59463ae3-8073-4b2d-ad87-fdb4486af77a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f1e192f3eab42ee9f9ed5d2789e0d1e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:16.884253 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59463ae3-8073-4b2d-ad87-fdb4486af77a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:17.131924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-59463ae3-8073-4b2d-ad87-fdb4486af77a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f1e192f3eab42ee9f9ed5d2789e0d1e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:17.133034 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:17.146774 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-59463ae3-8073-4b2d-ad87-fdb4486af77a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f1e192f3eab42ee9f9ed5d2789e0d1e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:17.235471 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-59463ae3-8073-4b2d-ad87-fdb4486af77a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f1e192f3eab42ee9f9ed5d2789e0d1e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:17.243633 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-59463ae3-8073-4b2d-ad87-fdb4486af77a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f1e192f3eab42ee9f9ed5d2789e0d1e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:17.244637 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-59463ae3-8073-4b2d-ad87-fdb4486af77a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f1e192f3eab42ee9f9ed5d2789e0d1e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:17.256157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1278/2534] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:02:16 2022] DELETE /identity/v3/users/9f1e192f3eab42ee9f9ed5d2789e0d1e => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:17.503561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467842c-4462-45ae-b945-19200b6344ca service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:17.515686 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467842c-4462-45ae-b945-19200b6344ca service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:17.555001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:17.597527 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18044b98-7774-4e79-9095-b1bc7e98862b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:17.598389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18044b98-7774-4e79-9095-b1bc7e98862b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:17.598671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18044b98-7774-4e79-9095-b1bc7e98862b admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:17.598943 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18044b98-7774-4e79-9095-b1bc7e98862b admin admin] PATH_INFO: `/v3/projects/a09fe8836f914998aec8ee2afcca9e16` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:17.603303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18044b98-7774-4e79-9095-b1bc7e98862b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a09fe8836f914998aec8ee2afcca9e16)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:17.603303 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18044b98-7774-4e79-9095-b1bc7e98862b admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:17.807001 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-18044b98-7774-4e79-9095-b1bc7e98862b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a09fe8836f914998aec8ee2afcca9e16', 'request_id': 'req-18044b98-7774-4e79-9095-b1bc7e98862b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:17.813724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-18044b98-7774-4e79-9095-b1bc7e98862b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a09fe8836f914998aec8ee2afcca9e16', 'request_id': 'req-18044b98-7774-4e79-9095-b1bc7e98862b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:17.885424 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-18044b98-7774-4e79-9095-b1bc7e98862b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a09fe8836f914998aec8ee2afcca9e16', 'request_id': 'req-18044b98-7774-4e79-9095-b1bc7e98862b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:17.896308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-18044b98-7774-4e79-9095-b1bc7e98862b admin admin] The token cache is being invalidate because project a09fe8836f914998aec8ee2afcca9e16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:02:17.896592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-18044b98-7774-4e79-9095-b1bc7e98862b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:17.897292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-18044b98-7774-4e79-9095-b1bc7e98862b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:17.898816 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1257/2535] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:02:17 2022] DELETE /identity/v3/projects/a09fe8836f914998aec8ee2afcca9e16 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:17.904121 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71f1db-64ca-42d7-8ac8-0e71e2dcb125 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:17.904584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71f1db-64ca-42d7-8ac8-0e71e2dcb125 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:17.904710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71f1db-64ca-42d7-8ac8-0e71e2dcb125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:17.922745 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-7e71f1db-64ca-42d7-8ac8-0e71e2dcb125 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:17.923956 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7e71f1db-64ca-42d7-8ac8-0e71e2dcb125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:17.983721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1279/2536] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:17.992230 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5076d-b911-46ee-9c81-8497eb378949 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:17.992523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5076d-b911-46ee-9c81-8497eb378949 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:17.993461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5076d-b911-46ee-9c81-8497eb378949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:18.011684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-73c5076d-b911-46ee-9c81-8497eb378949 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:18.012893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-73c5076d-b911-46ee-9c81-8497eb378949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:18.021022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1258/2537] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:18.025612 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71f1db-64ca-42d7-8ac8-0e71e2dcb125 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:18.035154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71f1db-64ca-42d7-8ac8-0e71e2dcb125 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:18.040249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71399951-d5a6-4f15-b9a9-877432df1567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:18.041290 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71399951-d5a6-4f15-b9a9-877432df1567 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:18.041581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71399951-d5a6-4f15-b9a9-877432df1567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:18.041861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71399951-d5a6-4f15-b9a9-877432df1567 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:18.042593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71399951-d5a6-4f15-b9a9-877432df1567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:02:18.042899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71399951-d5a6-4f15-b9a9-877432df1567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:18.044490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71399951-d5a6-4f15-b9a9-877432df1567 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:18.057103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1280/2538] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:02:18.061777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5076d-b911-46ee-9c81-8497eb378949 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:18.071392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5076d-b911-46ee-9c81-8497eb378949 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:18.076588 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535976e-7106-4731-9846-97fa0272f86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:18.077277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535976e-7106-4731-9846-97fa0272f86c admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:18.077559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535976e-7106-4731-9846-97fa0272f86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:18.077832 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535976e-7106-4731-9846-97fa0272f86c admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:18.078519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535976e-7106-4731-9846-97fa0272f86c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:18.080524 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535976e-7106-4731-9846-97fa0272f86c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:18.172967 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1259/2539] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:02:18 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:02:18.177094 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71399951-d5a6-4f15-b9a9-877432df1567 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:18.188068 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71399951-d5a6-4f15-b9a9-877432df1567 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:18.248349 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ba386-0224-4621-9305-6406c7b68c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:18.248975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ba386-0224-4621-9305-6406c7b68c93 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:18.249274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ba386-0224-4621-9305-6406c7b68c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:18.249558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ba386-0224-4621-9305-6406c7b68c93 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:18.250269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ba386-0224-4621-9305-6406c7b68c93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:18.252212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ba386-0224-4621-9305-6406c7b68c93 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:18.342989 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1281/2540] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:02:18 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:02:18.348320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535976e-7106-4731-9846-97fa0272f86c admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:18.359650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535976e-7106-4731-9846-97fa0272f86c admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:18.364515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc2fe7-ea8c-40e5-ab7d-080676edd253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:18.365198 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc2fe7-ea8c-40e5-ab7d-080676edd253 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:18.365479 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc2fe7-ea8c-40e5-ab7d-080676edd253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:18.365752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc2fe7-ea8c-40e5-ab7d-080676edd253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:18.366483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc2fe7-ea8c-40e5-ab7d-080676edd253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:18.368098 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc2fe7-ea8c-40e5-ab7d-080676edd253 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:18.379928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1260/2541] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:18.383945 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ba386-0224-4621-9305-6406c7b68c93 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:18.394587 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ba386-0224-4621-9305-6406c7b68c93 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:18.399966 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff4828-ac66-478d-b9ab-2cb00d132b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:18.400639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff4828-ac66-478d-b9ab-2cb00d132b5e admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:18.400924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff4828-ac66-478d-b9ab-2cb00d132b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:18.402781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff4828-ac66-478d-b9ab-2cb00d132b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:18.402781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff4828-ac66-478d-b9ab-2cb00d132b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:18.403544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff4828-ac66-478d-b9ab-2cb00d132b5e admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:18.413776 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1282/2542] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:18.418659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc2fe7-ea8c-40e5-ab7d-080676edd253 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:18.428782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc2fe7-ea8c-40e5-ab7d-080676edd253 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:18.433739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe6909-a094-48c9-a9fe-93da63119059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:18.434614 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe6909-a094-48c9-a9fe-93da63119059 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:18.434894 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe6909-a094-48c9-a9fe-93da63119059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:18.435169 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe6909-a094-48c9-a9fe-93da63119059 admin admin] PATH_INFO: `/v3/projects/3e290776a3b640dda072c7b129799621/users/69b3cc21829c4ac88a63bbb9ffd39a86/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:18.449999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe6909-a094-48c9-a9fe-93da63119059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e290776a3b640dda072c7b129799621, user_id=69b3cc21829c4ac88a63bbb9ffd39a86, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:18.452633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe6909-a094-48c9-a9fe-93da63119059 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:18.567781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1261/2543] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:18 2022] PUT /identity/v3/projects/3e290776a3b640dda072c7b129799621/users/69b3cc21829c4ac88a63bbb9ffd39a86/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:18.578384 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b29ea-e61c-4908-9f87-66d0b55a505b None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:18.578716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b29ea-e61c-4908-9f87-66d0b55a505b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:18.579016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b29ea-e61c-4908-9f87-66d0b55a505b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:18.607272 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-162b29ea-e61c-4908-9f87-66d0b55a505b None None] MFA Rules not processed for user `69b3cc21829c4ac88a63bbb9ffd39a86`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:18.608905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-162b29ea-e61c-4908-9f87-66d0b55a505b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:18.629554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:18.677812 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1283/2544] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:18.689176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe6909-a094-48c9-a9fe-93da63119059 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:18.690870 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a3fe6909-a094-48c9-a9fe-93da63119059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:18.742439 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:18.768197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe6909-a094-48c9-a9fe-93da63119059 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:18.774146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa059d-a0eb-4620-a5a9-b4f9ca293ba6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:18.774839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa059d-a0eb-4620-a5a9-b4f9ca293ba6 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:18.775520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa059d-a0eb-4620-a5a9-b4f9ca293ba6 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:18.775994 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa059d-a0eb-4620-a5a9-b4f9ca293ba6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:18.787072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa059d-a0eb-4620-a5a9-b4f9ca293ba6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:18.788562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa059d-a0eb-4620-a5a9-b4f9ca293ba6 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:18.799345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1262/2545] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:19.012075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b29ea-e61c-4908-9f87-66d0b55a505b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:19.078696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b29ea-e61c-4908-9f87-66d0b55a505b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:19.083358 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae8b54-55b1-435e-b136-32ac15426c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:19.084232 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae8b54-55b1-435e-b136-32ac15426c84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:19.084536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae8b54-55b1-435e-b136-32ac15426c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:19.084848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae8b54-55b1-435e-b136-32ac15426c84 admin admin] PATH_INFO: `/v3/users/69b3cc21829c4ac88a63bbb9ffd39a86` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:19.086063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae8b54-55b1-435e-b136-32ac15426c84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69b3cc21829c4ac88a63bbb9ffd39a86)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:19.088875 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae8b54-55b1-435e-b136-32ac15426c84 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:19.279503 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-85ae8b54-55b1-435e-b136-32ac15426c84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69b3cc21829c4ac88a63bbb9ffd39a86'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:19.296138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-85ae8b54-55b1-435e-b136-32ac15426c84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69b3cc21829c4ac88a63bbb9ffd39a86'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:19.378658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-85ae8b54-55b1-435e-b136-32ac15426c84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69b3cc21829c4ac88a63bbb9ffd39a86'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:19.389581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-85ae8b54-55b1-435e-b136-32ac15426c84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69b3cc21829c4ac88a63bbb9ffd39a86'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:19.390833 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-85ae8b54-55b1-435e-b136-32ac15426c84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69b3cc21829c4ac88a63bbb9ffd39a86'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:19.398203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1284/2546] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:02:19 2022] DELETE /identity/v3/users/69b3cc21829c4ac88a63bbb9ffd39a86 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:19.410239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f32f5-b063-4270-8960-df4406a424cf None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:19.410239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f32f5-b063-4270-8960-df4406a424cf None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:19.410239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f32f5-b063-4270-8960-df4406a424cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:19.429929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-f62f32f5-b063-4270-8960-df4406a424cf None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:19.432443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f62f32f5-b063-4270-8960-df4406a424cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:19.509209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1263/2547] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:19.518722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae8b54-55b1-435e-b136-32ac15426c84 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:19.520229 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-85ae8b54-55b1-435e-b136-32ac15426c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:19.574560 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:19.589196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae8b54-55b1-435e-b136-32ac15426c84 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:19.592748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651dc798-83dc-49fa-a7ab-a5c2ff313f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:19.593462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dc798-83dc-49fa-a7ab-a5c2ff313f3b service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:19.593748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dc798-83dc-49fa-a7ab-a5c2ff313f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:19.594030 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dc798-83dc-49fa-a7ab-a5c2ff313f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:19.603406 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651dc798-83dc-49fa-a7ab-a5c2ff313f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:19.605205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651dc798-83dc-49fa-a7ab-a5c2ff313f3b service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:19.615138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1285/2548] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:20.046820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f32f5-b063-4270-8960-df4406a424cf None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:20.048264 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f62f32f5-b063-4270-8960-df4406a424cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:20.072469 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f32f5-b063-4270-8960-df4406a424cf None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:20.077316 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed57c62-8952-4b8f-9c15-5cb6cd3717da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:20.078229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed57c62-8952-4b8f-9c15-5cb6cd3717da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:20.078506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed57c62-8952-4b8f-9c15-5cb6cd3717da admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:20.078781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed57c62-8952-4b8f-9c15-5cb6cd3717da admin admin] PATH_INFO: `/v3/projects/3e290776a3b640dda072c7b129799621` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:20.080084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed57c62-8952-4b8f-9c15-5cb6cd3717da admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e290776a3b640dda072c7b129799621)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:20.082110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed57c62-8952-4b8f-9c15-5cb6cd3717da admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:20.188756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-5ed57c62-8952-4b8f-9c15-5cb6cd3717da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e290776a3b640dda072c7b129799621', 'request_id': 'req-5ed57c62-8952-4b8f-9c15-5cb6cd3717da'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:20.195358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-5ed57c62-8952-4b8f-9c15-5cb6cd3717da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e290776a3b640dda072c7b129799621', 'request_id': 'req-5ed57c62-8952-4b8f-9c15-5cb6cd3717da'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:20.307965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-5ed57c62-8952-4b8f-9c15-5cb6cd3717da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e290776a3b640dda072c7b129799621', 'request_id': 'req-5ed57c62-8952-4b8f-9c15-5cb6cd3717da'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:20.319036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-5ed57c62-8952-4b8f-9c15-5cb6cd3717da admin admin] The token cache is being invalidate because project 3e290776a3b640dda072c7b129799621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:02:20.319319 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-5ed57c62-8952-4b8f-9c15-5cb6cd3717da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:20.319962 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-5ed57c62-8952-4b8f-9c15-5cb6cd3717da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:20.322109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1264/2549] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:02:20 2022] DELETE /identity/v3/projects/3e290776a3b640dda072c7b129799621 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:20.327173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8fadc-c5c6-490c-bc62-e12366fcebb9 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:20.327640 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8fadc-c5c6-490c-bc62-e12366fcebb9 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:20.327805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8fadc-c5c6-490c-bc62-e12366fcebb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:20.346953 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-14c8fadc-c5c6-490c-bc62-e12366fcebb9 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:20.348144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-14c8fadc-c5c6-490c-bc62-e12366fcebb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:20.409727 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1286/2550] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:20.418462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce12e2-7f72-43cf-8d1e-44a79095dd59 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:20.418762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce12e2-7f72-43cf-8d1e-44a79095dd59 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:20.419049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce12e2-7f72-43cf-8d1e-44a79095dd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:20.438749 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-f4ce12e2-7f72-43cf-8d1e-44a79095dd59 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:20.440010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f4ce12e2-7f72-43cf-8d1e-44a79095dd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:20.449361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1265/2551] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:20.454571 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8fadc-c5c6-490c-bc62-e12366fcebb9 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:20.465256 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8fadc-c5c6-490c-bc62-e12366fcebb9 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:20.470562 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdbbc2-730c-4b4a-9a78-602ee3ed4373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:20.471318 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdbbc2-730c-4b4a-9a78-602ee3ed4373 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:20.471625 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdbbc2-730c-4b4a-9a78-602ee3ed4373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:20.471893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdbbc2-730c-4b4a-9a78-602ee3ed4373 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:20.472740 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdbbc2-730c-4b4a-9a78-602ee3ed4373 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:02:20.473086 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdbbc2-730c-4b4a-9a78-602ee3ed4373 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:20.474791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdbbc2-730c-4b4a-9a78-602ee3ed4373 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:20.490320 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1287/2552] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:02:20.500717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce12e2-7f72-43cf-8d1e-44a79095dd59 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:20.513334 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce12e2-7f72-43cf-8d1e-44a79095dd59 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:20.519238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6d3b8-aec7-4197-909f-52f8e1bf0343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:20.520043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d3b8-aec7-4197-909f-52f8e1bf0343 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:20.520201 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d3b8-aec7-4197-909f-52f8e1bf0343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:20.520492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d3b8-aec7-4197-909f-52f8e1bf0343 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:20.521547 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6d3b8-aec7-4197-909f-52f8e1bf0343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:20.523146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6d3b8-aec7-4197-909f-52f8e1bf0343 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:20.644259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1266/2553] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:02:20 2022] POST /identity/v3/projects => generated 405 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:02:20.648368 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdbbc2-730c-4b4a-9a78-602ee3ed4373 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:20.658663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdbbc2-730c-4b4a-9a78-602ee3ed4373 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:20.715606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf831f5-10a4-4812-8a6c-0a2927577760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:20.716269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf831f5-10a4-4812-8a6c-0a2927577760 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:20.716549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf831f5-10a4-4812-8a6c-0a2927577760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:20.716821 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf831f5-10a4-4812-8a6c-0a2927577760 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:20.717554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf831f5-10a4-4812-8a6c-0a2927577760 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:20.719474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf831f5-10a4-4812-8a6c-0a2927577760 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:20.802083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1288/2554] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:02:20 2022] POST /identity/v3/users => generated 303 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:02:20.806117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6d3b8-aec7-4197-909f-52f8e1bf0343 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:20.818296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6d3b8-aec7-4197-909f-52f8e1bf0343 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:20.826434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf831f5-10a4-4812-8a6c-0a2927577760 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:20.827110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683aac71-dff9-43fd-922c-4ec4396ad99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:20.827792 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683aac71-dff9-43fd-922c-4ec4396ad99d admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:20.828158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683aac71-dff9-43fd-922c-4ec4396ad99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:20.828264 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2bf831f5-10a4-4812-8a6c-0a2927577760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:20.828437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683aac71-dff9-43fd-922c-4ec4396ad99d admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:20.829218 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683aac71-dff9-43fd-922c-4ec4396ad99d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:20.830910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683aac71-dff9-43fd-922c-4ec4396ad99d admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:20.841915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf831f5-10a4-4812-8a6c-0a2927577760 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:20.843287 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1267/2555] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:20.847305 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683aac71-dff9-43fd-922c-4ec4396ad99d admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:20.849749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e18f09-18d1-4939-a2c3-891947fc3d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:20.850639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e18f09-18d1-4939-a2c3-891947fc3d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:20.850926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e18f09-18d1-4939-a2c3-891947fc3d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:20.851205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e18f09-18d1-4939-a2c3-891947fc3d41 admin admin] PATH_INFO: `/v3/users/68e854ada59343f2bea86961e83eda4e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:20.852481 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e18f09-18d1-4939-a2c3-891947fc3d41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68e854ada59343f2bea86961e83eda4e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:20.855184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e18f09-18d1-4939-a2c3-891947fc3d41 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:20.858156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683aac71-dff9-43fd-922c-4ec4396ad99d admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:20.862845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03c7ad-cab0-4898-9ad7-0082f916af6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:20.863495 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03c7ad-cab0-4898-9ad7-0082f916af6b admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:20.863776 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03c7ad-cab0-4898-9ad7-0082f916af6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:20.864049 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03c7ad-cab0-4898-9ad7-0082f916af6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:20.864775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03c7ad-cab0-4898-9ad7-0082f916af6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:20.866403 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03c7ad-cab0-4898-9ad7-0082f916af6b admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:20.877879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1268/2556] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:20.882233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03c7ad-cab0-4898-9ad7-0082f916af6b admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:20.888921 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:20.892826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03c7ad-cab0-4898-9ad7-0082f916af6b admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:20.897792 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dee02-bfbe-4e3f-9fca-30e061268553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:20.898672 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dee02-bfbe-4e3f-9fca-30e061268553 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:20.898956 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dee02-bfbe-4e3f-9fca-30e061268553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:20.899233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dee02-bfbe-4e3f-9fca-30e061268553 admin admin] PATH_INFO: `/v3/projects/0966f704811c4a4f8c98ec4f864cf189/users/78a6a3fa055b44e09aa02a1df2e361f7/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:20.913799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dee02-bfbe-4e3f-9fca-30e061268553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0966f704811c4a4f8c98ec4f864cf189, user_id=78a6a3fa055b44e09aa02a1df2e361f7, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:20.917061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dee02-bfbe-4e3f-9fca-30e061268553 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:21.014310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1269/2557] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:20 2022] PUT /identity/v3/projects/0966f704811c4a4f8c98ec4f864cf189/users/78a6a3fa055b44e09aa02a1df2e361f7/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:02:21.025984 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110e4fa-587f-44c7-b8c1-41716fab4a82 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:21.026274 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110e4fa-587f-44c7-b8c1-41716fab4a82 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:21.026554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110e4fa-587f-44c7-b8c1-41716fab4a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:21.052592 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b110e4fa-587f-44c7-b8c1-41716fab4a82 None None] MFA Rules not processed for user `78a6a3fa055b44e09aa02a1df2e361f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:21.053830 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b110e4fa-587f-44c7-b8c1-41716fab4a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:21.082070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-b3e18f09-18d1-4939-a2c3-891947fc3d41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68e854ada59343f2bea86961e83eda4e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:21.087500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:21.095049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-b3e18f09-18d1-4939-a2c3-891947fc3d41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68e854ada59343f2bea86961e83eda4e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:21.119897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1270/2558] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:21 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:21.124385 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110e4fa-587f-44c7-b8c1-41716fab4a82 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:21.187389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-b3e18f09-18d1-4939-a2c3-891947fc3d41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68e854ada59343f2bea86961e83eda4e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:21.187693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b110e4fa-587f-44c7-b8c1-41716fab4a82 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:21.192306 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd2630-db81-438a-8f2c-f3fc0faafc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:21.193097 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd2630-db81-438a-8f2c-f3fc0faafc92 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:21.193276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd2630-db81-438a-8f2c-f3fc0faafc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:21.193551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd2630-db81-438a-8f2c-f3fc0faafc92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:21.194247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd2630-db81-438a-8f2c-f3fc0faafc92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:21.195660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-b3e18f09-18d1-4939-a2c3-891947fc3d41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68e854ada59343f2bea86961e83eda4e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:21.196170 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd2630-db81-438a-8f2c-f3fc0faafc92 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:21.196421 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-b3e18f09-18d1-4939-a2c3-891947fc3d41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68e854ada59343f2bea86961e83eda4e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:21.203470 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1289/2559] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:02:20 2022] DELETE /identity/v3/users/68e854ada59343f2bea86961e83eda4e => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:21.207905 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea19ab-e73c-449e-a323-a34c1106a1f7 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:21.208193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea19ab-e73c-449e-a323-a34c1106a1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:21.208468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea19ab-e73c-449e-a323-a34c1106a1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:21.227395 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-b5ea19ab-e73c-449e-a323-a34c1106a1f7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:21.229386 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b5ea19ab-e73c-449e-a323-a34c1106a1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:21.284056 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1271/2560] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:02:21 2022] POST /identity/v3/projects => generated 403 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:02:21.287762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd2630-db81-438a-8f2c-f3fc0faafc92 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:21.289364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4dbd2630-db81-438a-8f2c-f3fc0faafc92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:21.290170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1290/2561] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:21.300603 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea19ab-e73c-449e-a323-a34c1106a1f7 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:21.302058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b5ea19ab-e73c-449e-a323-a34c1106a1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:21.351978 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:21.357700 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd2630-db81-438a-8f2c-f3fc0faafc92 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:21.361527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea19ab-e73c-449e-a323-a34c1106a1f7 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:21.362724 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5467af-445f-4979-af09-25bdd32d7bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:21.363405 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5467af-445f-4979-af09-25bdd32d7bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:21.363686 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5467af-445f-4979-af09-25bdd32d7bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:21.363972 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5467af-445f-4979-af09-25bdd32d7bc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:21.364665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5467af-445f-4979-af09-25bdd32d7bc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:21.366639 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5467af-445f-4979-af09-25bdd32d7bc0 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:21.409865 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef3aad-3c9c-4d2e-aaa4-c4ea1c904e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:21.410514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef3aad-3c9c-4d2e-aaa4-c4ea1c904e0a service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:21.410801 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef3aad-3c9c-4d2e-aaa4-c4ea1c904e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:21.411079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef3aad-3c9c-4d2e-aaa4-c4ea1c904e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:21.420873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef3aad-3c9c-4d2e-aaa4-c4ea1c904e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:21.422343 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef3aad-3c9c-4d2e-aaa4-c4ea1c904e0a service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:21.432073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1291/2562] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 11:02:21.444699 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1272/2563] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:02:21 2022] POST /identity/v3/users => generated 308 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:02:21.445912 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef3aad-3c9c-4d2e-aaa4-c4ea1c904e0a service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:21.456642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef3aad-3c9c-4d2e-aaa4-c4ea1c904e0a service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:21.463446 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f870bd0-bf1a-469f-86b5-169583baec5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:21.464351 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f870bd0-bf1a-469f-86b5-169583baec5a admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:21.464636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f870bd0-bf1a-469f-86b5-169583baec5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:21.464919 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f870bd0-bf1a-469f-86b5-169583baec5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:21.465818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f870bd0-bf1a-469f-86b5-169583baec5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:21.467459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f870bd0-bf1a-469f-86b5-169583baec5a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:21.479274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1292/2564] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:21.483842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5467af-445f-4979-af09-25bdd32d7bc0 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:21.494343 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5467af-445f-4979-af09-25bdd32d7bc0 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:21.499520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607322ff-dc40-405d-997f-2f7c309d07c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:21.500522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607322ff-dc40-405d-997f-2f7c309d07c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:21.500887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607322ff-dc40-405d-997f-2f7c309d07c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:21.501215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607322ff-dc40-405d-997f-2f7c309d07c7 admin admin] PATH_INFO: `/v3/projects/45bcd943e43e4ff5a5362c60c6a5dd14/users/d00705882a2b43e2a454abf63cfdb29d/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:21.519157 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607322ff-dc40-405d-997f-2f7c309d07c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45bcd943e43e4ff5a5362c60c6a5dd14, user_id=d00705882a2b43e2a454abf63cfdb29d, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:21.522006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607322ff-dc40-405d-997f-2f7c309d07c7 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:21.605873 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1273/2565] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:21 2022] PUT /identity/v3/projects/45bcd943e43e4ff5a5362c60c6a5dd14/users/d00705882a2b43e2a454abf63cfdb29d/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:02:21.610197 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f870bd0-bf1a-469f-86b5-169583baec5a admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:21.620608 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f870bd0-bf1a-469f-86b5-169583baec5a admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:21.638927 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:21.687665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfadaa-2607-4cc5-b955-edad8a522cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:21.688856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfadaa-2607-4cc5-b955-edad8a522cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:21.689420 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfadaa-2607-4cc5-b955-edad8a522cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:21.689932 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfadaa-2607-4cc5-b955-edad8a522cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:21.691416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfadaa-2607-4cc5-b955-edad8a522cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:21.695320 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfadaa-2607-4cc5-b955-edad8a522cb3 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:21.716861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1293/2566] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:21.721526 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607322ff-dc40-405d-997f-2f7c309d07c7 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:21.734494 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607322ff-dc40-405d-997f-2f7c309d07c7 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:21.739599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad720f9-4868-46b7-80db-993d2c9bc088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:21.740514 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad720f9-4868-46b7-80db-993d2c9bc088 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:21.741085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad720f9-4868-46b7-80db-993d2c9bc088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:21.741375 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad720f9-4868-46b7-80db-993d2c9bc088 admin admin] PATH_INFO: `/v3/projects/45bcd943e43e4ff5a5362c60c6a5dd14/users/d00705882a2b43e2a454abf63cfdb29d/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:21.743429 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad720f9-4868-46b7-80db-993d2c9bc088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45bcd943e43e4ff5a5362c60c6a5dd14, user_id=d00705882a2b43e2a454abf63cfdb29d, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:21.746400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad720f9-4868-46b7-80db-993d2c9bc088 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:21.758333 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1274/2567] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:21 2022] PUT /identity/v3/projects/45bcd943e43e4ff5a5362c60c6a5dd14/users/d00705882a2b43e2a454abf63cfdb29d/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:21.769234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4d5d7-6870-4957-83c7-8f75a4ece2c9 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:21.769543 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4d5d7-6870-4957-83c7-8f75a4ece2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:21.769807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4d5d7-6870-4957-83c7-8f75a4ece2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:21.799259 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-7cb4d5d7-6870-4957-83c7-8f75a4ece2c9 None None] MFA Rules not processed for user `d00705882a2b43e2a454abf63cfdb29d`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:21.800416 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7cb4d5d7-6870-4957-83c7-8f75a4ece2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:21.878962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1294/2568] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:21 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:21.890944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad720f9-4868-46b7-80db-993d2c9bc088 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:21.892766 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4ad720f9-4868-46b7-80db-993d2c9bc088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:21.938641 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4d5d7-6870-4957-83c7-8f75a4ece2c9 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:21.940190 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7cb4d5d7-6870-4957-83c7-8f75a4ece2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:21.954911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad720f9-4868-46b7-80db-993d2c9bc088 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:21.958883 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64903a1b-0d98-478d-b51f-c602561f1c00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:21.959701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64903a1b-0d98-478d-b51f-c602561f1c00 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:21.960092 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64903a1b-0d98-478d-b51f-c602561f1c00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:21.960332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64903a1b-0d98-478d-b51f-c602561f1c00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:21.971251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64903a1b-0d98-478d-b51f-c602561f1c00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:21.972739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64903a1b-0d98-478d-b51f-c602561f1c00 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:21.984907 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1275/2569] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:21 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:22.022079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4d5d7-6870-4957-83c7-8f75a4ece2c9 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:22.029563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa4697-c09f-40c4-b7a5-6ccc0b58e341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:22.030494 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa4697-c09f-40c4-b7a5-6ccc0b58e341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:22.030810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa4697-c09f-40c4-b7a5-6ccc0b58e341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:22.031143 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa4697-c09f-40c4-b7a5-6ccc0b58e341 admin admin] PATH_INFO: `/v3/projects/f01e7ea926544200a89bc6554fe7ad11` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:22.032453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa4697-c09f-40c4-b7a5-6ccc0b58e341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f01e7ea926544200a89bc6554fe7ad11)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:22.034433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa4697-c09f-40c4-b7a5-6ccc0b58e341 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:22.127564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64903a1b-0d98-478d-b51f-c602561f1c00 service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:22.138894 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64903a1b-0d98-478d-b51f-c602561f1c00 service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:22.149979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:22.154466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a8aa4697-c09f-40c4-b7a5-6ccc0b58e341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f01e7ea926544200a89bc6554fe7ad11', 'request_id': 'req-a8aa4697-c09f-40c4-b7a5-6ccc0b58e341'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:22.161228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a8aa4697-c09f-40c4-b7a5-6ccc0b58e341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f01e7ea926544200a89bc6554fe7ad11', 'request_id': 'req-a8aa4697-c09f-40c4-b7a5-6ccc0b58e341'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:22.167419 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a8aa4697-c09f-40c4-b7a5-6ccc0b58e341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f01e7ea926544200a89bc6554fe7ad11', 'request_id': 'req-a8aa4697-c09f-40c4-b7a5-6ccc0b58e341'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:22.199825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a97856-de8a-43a6-8483-9b432b33eb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:22.200733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a97856-de8a-43a6-8483-9b432b33eb00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:22.201057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a97856-de8a-43a6-8483-9b432b33eb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:22.201338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a97856-de8a-43a6-8483-9b432b33eb00 admin admin] PATH_INFO: `/v3/users/78a6a3fa055b44e09aa02a1df2e361f7` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:22.202535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a97856-de8a-43a6-8483-9b432b33eb00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a6a3fa055b44e09aa02a1df2e361f7)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:22.204522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a97856-de8a-43a6-8483-9b432b33eb00 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:22.233432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a8aa4697-c09f-40c4-b7a5-6ccc0b58e341 admin admin] The token cache is being invalidate because project f01e7ea926544200a89bc6554fe7ad11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:02:22.233719 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a8aa4697-c09f-40c4-b7a5-6ccc0b58e341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:22.234364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a8aa4697-c09f-40c4-b7a5-6ccc0b58e341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:22.236099 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1295/2570] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:02:21 2022] DELETE /identity/v3/projects/f01e7ea926544200a89bc6554fe7ad11 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:02:22.240939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec694929-ce51-42af-9375-a3c236de0946 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:22.241268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec694929-ce51-42af-9375-a3c236de0946 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:22.241553 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec694929-ce51-42af-9375-a3c236de0946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:22.261870 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-ec694929-ce51-42af-9375-a3c236de0946 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:22.263103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ec694929-ce51-42af-9375-a3c236de0946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:22.272217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1296/2571] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:22.279967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed3bee-090d-44a1-9845-78dcd9d8e0c0 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:22.280251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed3bee-090d-44a1-9845-78dcd9d8e0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:22.280528 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed3bee-090d-44a1-9845-78dcd9d8e0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:22.299090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-11ed3bee-090d-44a1-9845-78dcd9d8e0c0 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:22.300446 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-11ed3bee-090d-44a1-9845-78dcd9d8e0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:22.309091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1297/2572] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:22.313090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed3bee-090d-44a1-9845-78dcd9d8e0c0 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:22.322237 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed3bee-090d-44a1-9845-78dcd9d8e0c0 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:22.326920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad72e67-48cc-48e6-9264-f26174a7c351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:22.327535 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad72e67-48cc-48e6-9264-f26174a7c351 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:22.327798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad72e67-48cc-48e6-9264-f26174a7c351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:22.328113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad72e67-48cc-48e6-9264-f26174a7c351 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:22.328840 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad72e67-48cc-48e6-9264-f26174a7c351 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:02:22.329178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad72e67-48cc-48e6-9264-f26174a7c351 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:22.330771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad72e67-48cc-48e6-9264-f26174a7c351 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:22.343283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1298/2573] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 11:02:22.347720 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad72e67-48cc-48e6-9264-f26174a7c351 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:22.358767 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad72e67-48cc-48e6-9264-f26174a7c351 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:22.360188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f4a97856-de8a-43a6-8483-9b432b33eb00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a6a3fa055b44e09aa02a1df2e361f7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:22.363873 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:22.364518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4cd32-b920-4522-80e0-be1997a1e005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:22.365403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4cd32-b920-4522-80e0-be1997a1e005 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:22.365691 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4cd32-b920-4522-80e0-be1997a1e005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:22.365967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4cd32-b920-4522-80e0-be1997a1e005 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:22.366657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4cd32-b920-4522-80e0-be1997a1e005 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:22.369233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4cd32-b920-4522-80e0-be1997a1e005 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:22.461794 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f4a97856-de8a-43a6-8483-9b432b33eb00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a6a3fa055b44e09aa02a1df2e361f7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:22.462542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f4a97856-de8a-43a6-8483-9b432b33eb00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a6a3fa055b44e09aa02a1df2e361f7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:22.463343 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f4a97856-de8a-43a6-8483-9b432b33eb00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a6a3fa055b44e09aa02a1df2e361f7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:22.475992 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f4a97856-de8a-43a6-8483-9b432b33eb00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a6a3fa055b44e09aa02a1df2e361f7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:22.489253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1276/2574] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:02:22 2022] DELETE /identity/v3/users/78a6a3fa055b44e09aa02a1df2e361f7 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:02:22.493523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372879b6-4ba7-4a70-b969-34703e303aba None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:22.494604 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372879b6-4ba7-4a70-b969-34703e303aba None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:22.494966 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372879b6-4ba7-4a70-b969-34703e303aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:22.503921 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1299/2575] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:02:22 2022] POST /identity/v3/projects => generated 371 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:02:22.510082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4cd32-b920-4522-80e0-be1997a1e005 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:22.511768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-bae4cd32-b920-4522-80e0-be1997a1e005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:22.515016 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-372879b6-4ba7-4a70-b969-34703e303aba None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:22.516192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-372879b6-4ba7-4a70-b969-34703e303aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:22.580498 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1277/2576] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:22.588981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4cd32-b920-4522-80e0-be1997a1e005 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:22.591765 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372879b6-4ba7-4a70-b969-34703e303aba None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:22.593928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-372879b6-4ba7-4a70-b969-34703e303aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:22.597392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78818b5a-e113-4616-a4b8-00f93fc13342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:22.598130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78818b5a-e113-4616-a4b8-00f93fc13342 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:22.598412 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78818b5a-e113-4616-a4b8-00f93fc13342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:22.598698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78818b5a-e113-4616-a4b8-00f93fc13342 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:22.599397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78818b5a-e113-4616-a4b8-00f93fc13342 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:22.601730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78818b5a-e113-4616-a4b8-00f93fc13342 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:22.654585 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372879b6-4ba7-4a70-b969-34703e303aba None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:22.658254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af3148-ad80-4fa5-b5b0-1c7265e6bbc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:22.658898 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af3148-ad80-4fa5-b5b0-1c7265e6bbc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:22.659182 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af3148-ad80-4fa5-b5b0-1c7265e6bbc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:22.659458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af3148-ad80-4fa5-b5b0-1c7265e6bbc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:22.669693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af3148-ad80-4fa5-b5b0-1c7265e6bbc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:22.671138 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af3148-ad80-4fa5-b5b0-1c7265e6bbc2 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:22.681498 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1278/2577] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:22.685167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1300/2578] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:02:22 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:02:22.686187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af3148-ad80-4fa5-b5b0-1c7265e6bbc2 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:22.695707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af3148-ad80-4fa5-b5b0-1c7265e6bbc2 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:22.702229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f2a9a-5b38-48c0-b790-75ae5b08e492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:22.702411 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f2a9a-5b38-48c0-b790-75ae5b08e492 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:22.702676 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f2a9a-5b38-48c0-b790-75ae5b08e492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:22.702875 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f2a9a-5b38-48c0-b790-75ae5b08e492 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:22.703645 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f2a9a-5b38-48c0-b790-75ae5b08e492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:22.705293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f2a9a-5b38-48c0-b790-75ae5b08e492 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:22.717000 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1279/2579] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:02:22.720547 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78818b5a-e113-4616-a4b8-00f93fc13342 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:22.731888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78818b5a-e113-4616-a4b8-00f93fc13342 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:22.737263 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22784b0c-e6ad-4227-b99c-23aed583d533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:22.737937 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22784b0c-e6ad-4227-b99c-23aed583d533 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:22.738287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22784b0c-e6ad-4227-b99c-23aed583d533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:22.738566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22784b0c-e6ad-4227-b99c-23aed583d533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:22.739305 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22784b0c-e6ad-4227-b99c-23aed583d533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:22.740985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22784b0c-e6ad-4227-b99c-23aed583d533 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:22.751925 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1301/2580] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:02:22.756127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f2a9a-5b38-48c0-b790-75ae5b08e492 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:22.766862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f2a9a-5b38-48c0-b790-75ae5b08e492 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:22.771923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434e7a0-adb2-423c-9c0f-cb29efb05c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:22.772919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e7a0-adb2-423c-9c0f-cb29efb05c6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:22.773232 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e7a0-adb2-423c-9c0f-cb29efb05c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:22.773510 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e7a0-adb2-423c-9c0f-cb29efb05c6b admin admin] PATH_INFO: `/v3/projects/c1b562ad06434d4590251665763b35b8/users/b78c09a127334196852ed9d8f67e4d07/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:22.789912 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434e7a0-adb2-423c-9c0f-cb29efb05c6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b562ad06434d4590251665763b35b8, user_id=b78c09a127334196852ed9d8f67e4d07, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:22.792695 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434e7a0-adb2-423c-9c0f-cb29efb05c6b admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:22.896873 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1280/2581] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:22 2022] PUT /identity/v3/projects/c1b562ad06434d4590251665763b35b8/users/b78c09a127334196852ed9d8f67e4d07/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:22.914695 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b175a-cb95-49e7-96d6-06e8ff72c8a1 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:22.917926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b175a-cb95-49e7-96d6-06e8ff72c8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:22.917926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1b175a-cb95-49e7-96d6-06e8ff72c8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:22.946031 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-ec1b175a-cb95-49e7-96d6-06e8ff72c8a1 None None] MFA Rules not processed for user `b78c09a127334196852ed9d8f67e4d07`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:22.948062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ec1b175a-cb95-49e7-96d6-06e8ff72c8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:23.000116 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:23.018425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1302/2582] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:23.023142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434e7a0-adb2-423c-9c0f-cb29efb05c6b admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:23.065055 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:23.085472 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434e7a0-adb2-423c-9c0f-cb29efb05c6b admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:23.090454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6e0be-ccb1-49ae-9052-0e95f3089163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:23.091109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6e0be-ccb1-49ae-9052-0e95f3089163 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:23.091387 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6e0be-ccb1-49ae-9052-0e95f3089163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:23.091661 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6e0be-ccb1-49ae-9052-0e95f3089163 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:23.092340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6e0be-ccb1-49ae-9052-0e95f3089163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:23.094256 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6e0be-ccb1-49ae-9052-0e95f3089163 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:23.204759 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b175a-cb95-49e7-96d6-06e8ff72c8a1 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:23.204759 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ec1b175a-cb95-49e7-96d6-06e8ff72c8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:23.218659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1b175a-cb95-49e7-96d6-06e8ff72c8a1 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:23.224924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:23.225832 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:23.227710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:23.227710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd admin admin] PATH_INFO: `/v3/projects/0966f704811c4a4f8c98ec4f864cf189` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:23.227710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0966f704811c4a4f8c98ec4f864cf189)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:23.230172 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:23.237362 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1281/2583] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:02:23 2022] POST /identity/v3/projects => generated 373 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:02:23.238450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6e0be-ccb1-49ae-9052-0e95f3089163 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:23.301800 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6e0be-ccb1-49ae-9052-0e95f3089163 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:23.306767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3452a7-2d00-425c-960b-87eefeaa14b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:23.307519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3452a7-2d00-425c-960b-87eefeaa14b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:23.307807 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3452a7-2d00-425c-960b-87eefeaa14b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:23.308113 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3452a7-2d00-425c-960b-87eefeaa14b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:23.308776 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3452a7-2d00-425c-960b-87eefeaa14b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:23.310735 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3452a7-2d00-425c-960b-87eefeaa14b6 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:23.350935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0966f704811c4a4f8c98ec4f864cf189', 'request_id': 'req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:23.357664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0966f704811c4a4f8c98ec4f864cf189', 'request_id': 'req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:23.363417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0966f704811c4a4f8c98ec4f864cf189', 'request_id': 'req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:23.408457 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1282/2584] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:02:23 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:02:23.412534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3452a7-2d00-425c-960b-87eefeaa14b6 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:23.423153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3452a7-2d00-425c-960b-87eefeaa14b6 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:23.435940 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd admin admin] The token cache is being invalidate because project 0966f704811c4a4f8c98ec4f864cf189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:02:23.436486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:23.436901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:23.438497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1303/2585] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:02:23 2022] DELETE /identity/v3/projects/0966f704811c4a4f8c98ec4f864cf189 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:02:23.442339 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:23.443837 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:23.487307 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c33908-89e2-4bc2-8833-7de5d51b7e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:23.487987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c33908-89e2-4bc2-8833-7de5d51b7e14 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:23.488267 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c33908-89e2-4bc2-8833-7de5d51b7e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:23.488538 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c33908-89e2-4bc2-8833-7de5d51b7e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:23.489415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c33908-89e2-4bc2-8833-7de5d51b7e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:23.491230 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c33908-89e2-4bc2-8833-7de5d51b7e14 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:23.506059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1283/2586] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:02:23.507149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c33908-89e2-4bc2-8833-7de5d51b7e14 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:23.508808 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-25c33908-89e2-4bc2-8833-7de5d51b7e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:23.514151 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6a534-0b5f-4ae0-baed-85e06f7cbadd admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:23.521319 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9a32c-dcca-4257-b536-39444ba67dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:23.521319 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9a32c-dcca-4257-b536-39444ba67dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:23.521600 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9a32c-dcca-4257-b536-39444ba67dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:23.521857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9a32c-dcca-4257-b536-39444ba67dd7 admin admin] PATH_INFO: `/v3/users/d00705882a2b43e2a454abf63cfdb29d` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:23.522344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c33908-89e2-4bc2-8833-7de5d51b7e14 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:23.527569 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9a32c-dcca-4257-b536-39444ba67dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d00705882a2b43e2a454abf63cfdb29d)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:23.528226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f73c24-2aa2-4476-9b44-c77e270e9e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:23.533619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f73c24-2aa2-4476-9b44-c77e270e9e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:23.533745 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9a32c-dcca-4257-b536-39444ba67dd7 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:23.533821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f73c24-2aa2-4476-9b44-c77e270e9e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:23.533821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f73c24-2aa2-4476-9b44-c77e270e9e6a admin admin] PATH_INFO: `/v3/projects/86bf824af415477a8cf8c9b3a799da30/users/c6c66c940def45d78318479521768afb/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:23.548334 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f73c24-2aa2-4476-9b44-c77e270e9e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=86bf824af415477a8cf8c9b3a799da30, user_id=c6c66c940def45d78318479521768afb, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:23.551104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f73c24-2aa2-4476-9b44-c77e270e9e6a admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:23.656739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1284/2587] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:23 2022] PUT /identity/v3/projects/86bf824af415477a8cf8c9b3a799da30/users/c6c66c940def45d78318479521768afb/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:23.660558 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f73c24-2aa2-4476-9b44-c77e270e9e6a admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:23.696107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:23.731664 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f73c24-2aa2-4476-9b44-c77e270e9e6a admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:23.736611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7143a-fb88-4187-b4f1-4de7ed87bcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:23.737389 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7143a-fb88-4187-b4f1-4de7ed87bcc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:23.737622 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7143a-fb88-4187-b4f1-4de7ed87bcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:23.737898 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7143a-fb88-4187-b4f1-4de7ed87bcc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:23.738635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7143a-fb88-4187-b4f1-4de7ed87bcc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:23.740243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7143a-fb88-4187-b4f1-4de7ed87bcc8 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:23.750767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1285/2588] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:02:23.754851 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7143a-fb88-4187-b4f1-4de7ed87bcc8 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:23.765571 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7143a-fb88-4187-b4f1-4de7ed87bcc8 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:23.770219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2672a-b411-4116-8a1a-ef2b9ec4f61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:23.771077 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2672a-b411-4116-8a1a-ef2b9ec4f61d admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:23.771373 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2672a-b411-4116-8a1a-ef2b9ec4f61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:23.771631 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2672a-b411-4116-8a1a-ef2b9ec4f61d admin admin] PATH_INFO: `/v3/projects/86bf824af415477a8cf8c9b3a799da30/users/c6c66c940def45d78318479521768afb/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:23.773601 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2672a-b411-4116-8a1a-ef2b9ec4f61d admin admin] RBAC: Authorizing `identity:create_grant(project_id=86bf824af415477a8cf8c9b3a799da30, user_id=c6c66c940def45d78318479521768afb, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:23.777031 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2672a-b411-4116-8a1a-ef2b9ec4f61d admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:23.787060 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1286/2589] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:23 2022] PUT /identity/v3/projects/86bf824af415477a8cf8c9b3a799da30/users/c6c66c940def45d78318479521768afb/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:02:23.798181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f2d70-ff07-4939-bc18-7a15abb0d7b8 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:23.798427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f2d70-ff07-4939-bc18-7a15abb0d7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:23.798712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f2d70-ff07-4939-bc18-7a15abb0d7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:23.821886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-49a9a32c-dcca-4257-b536-39444ba67dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd00705882a2b43e2a454abf63cfdb29d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:23.827218 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a57f2d70-ff07-4939-bc18-7a15abb0d7b8 None None] MFA Rules not processed for user `c6c66c940def45d78318479521768afb`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:23.828718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a57f2d70-ff07-4939-bc18-7a15abb0d7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:23.837605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-49a9a32c-dcca-4257-b536-39444ba67dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd00705882a2b43e2a454abf63cfdb29d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:23.903275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1287/2590] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:23.914527 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f2d70-ff07-4939-bc18-7a15abb0d7b8 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:23.916408 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a57f2d70-ff07-4939-bc18-7a15abb0d7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:23.921269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-49a9a32c-dcca-4257-b536-39444ba67dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd00705882a2b43e2a454abf63cfdb29d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:23.926467 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:23.929413 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-49a9a32c-dcca-4257-b536-39444ba67dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd00705882a2b43e2a454abf63cfdb29d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:23.930231 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-49a9a32c-dcca-4257-b536-39444ba67dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd00705882a2b43e2a454abf63cfdb29d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:23.936861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1304/2591] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:02:23 2022] DELETE /identity/v3/users/d00705882a2b43e2a454abf63cfdb29d => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:02:23.978641 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f2d70-ff07-4939-bc18-7a15abb0d7b8 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:24.034993 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:24.039580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b93694-997e-4ee0-b954-f85c787aa34e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:24.040403 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b93694-997e-4ee0-b954-f85c787aa34e service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:24.040837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b93694-997e-4ee0-b954-f85c787aa34e service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:24.041006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b93694-997e-4ee0-b954-f85c787aa34e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:24.042983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d8b93694-997e-4ee0-b954-f85c787aa34e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:24.110524 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b93694-997e-4ee0-b954-f85c787aa34e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:24.111963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b93694-997e-4ee0-b954-f85c787aa34e service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:24.122480 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1288/2592] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 11:02:24.359649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9a32c-dcca-4257-b536-39444ba67dd7 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:24.361107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-49a9a32c-dcca-4257-b536-39444ba67dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:24.373895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9a32c-dcca-4257-b536-39444ba67dd7 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:24.377899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b312c1-2888-41e0-864a-3f5903368008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:24.378566 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b312c1-2888-41e0-864a-3f5903368008 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:24.378848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b312c1-2888-41e0-864a-3f5903368008 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:24.379157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b312c1-2888-41e0-864a-3f5903368008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:24.380904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d9b312c1-2888-41e0-864a-3f5903368008 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:24.432798 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b93694-997e-4ee0-b954-f85c787aa34e service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:24.434353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d8b93694-997e-4ee0-b954-f85c787aa34e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:24.452651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b312c1-2888-41e0-864a-3f5903368008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:24.454217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b312c1-2888-41e0-864a-3f5903368008 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:24.465013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1305/2593] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:24.522453 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b93694-997e-4ee0-b954-f85c787aa34e service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:24.527454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e0208-b023-4ce8-bbc2-9ab9812767e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:24.528326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e0208-b023-4ce8-bbc2-9ab9812767e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:24.528608 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e0208-b023-4ce8-bbc2-9ab9812767e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:24.528943 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e0208-b023-4ce8-bbc2-9ab9812767e7 admin admin] PATH_INFO: `/v3/projects/45bcd943e43e4ff5a5362c60c6a5dd14` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:24.530273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e0208-b023-4ce8-bbc2-9ab9812767e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45bcd943e43e4ff5a5362c60c6a5dd14)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:24.532094 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e0208-b023-4ce8-bbc2-9ab9812767e7 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:24.665990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6b9e0208-b023-4ce8-bbc2-9ab9812767e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45bcd943e43e4ff5a5362c60c6a5dd14', 'request_id': 'req-6b9e0208-b023-4ce8-bbc2-9ab9812767e7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:24.672201 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6b9e0208-b023-4ce8-bbc2-9ab9812767e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45bcd943e43e4ff5a5362c60c6a5dd14', 'request_id': 'req-6b9e0208-b023-4ce8-bbc2-9ab9812767e7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:24.767117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6b9e0208-b023-4ce8-bbc2-9ab9812767e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45bcd943e43e4ff5a5362c60c6a5dd14', 'request_id': 'req-6b9e0208-b023-4ce8-bbc2-9ab9812767e7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:24.778925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6b9e0208-b023-4ce8-bbc2-9ab9812767e7 admin admin] The token cache is being invalidate because project 45bcd943e43e4ff5a5362c60c6a5dd14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:02:24.779209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6b9e0208-b023-4ce8-bbc2-9ab9812767e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:24.779839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6b9e0208-b023-4ce8-bbc2-9ab9812767e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:24.781411 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1289/2594] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:02:24 2022] DELETE /identity/v3/projects/45bcd943e43e4ff5a5362c60c6a5dd14 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:02:24.786334 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4ca3-5704-4e19-8713-09144e9fa47b None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:24.786637 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4ca3-5704-4e19-8713-09144e9fa47b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:24.786897 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa4ca3-5704-4e19-8713-09144e9fa47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:24.805378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-cffa4ca3-5704-4e19-8713-09144e9fa47b None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:24.809500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-cffa4ca3-5704-4e19-8713-09144e9fa47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:24.869619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1306/2595] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:24.877454 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf614b-dd72-4e91-bea1-a61921d49bf6 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:24.877750 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf614b-dd72-4e91-bea1-a61921d49bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:24.878030 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf614b-dd72-4e91-bea1-a61921d49bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:24.897605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a4bf614b-dd72-4e91-bea1-a61921d49bf6 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:24.898800 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a4bf614b-dd72-4e91-bea1-a61921d49bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:24.907956 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1290/2596] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:24.915017 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4ca3-5704-4e19-8713-09144e9fa47b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:24.925410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa4ca3-5704-4e19-8713-09144e9fa47b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:24.929941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c23253-d58a-4f0f-a73c-a246089e4b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:24.930593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c23253-d58a-4f0f-a73c-a246089e4b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:24.930874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c23253-d58a-4f0f-a73c-a246089e4b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:24.931673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c23253-d58a-4f0f-a73c-a246089e4b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:24.933057 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c23253-d58a-4f0f-a73c-a246089e4b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:02:24.933369 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c23253-d58a-4f0f-a73c-a246089e4b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:24.934973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c23253-d58a-4f0f-a73c-a246089e4b3a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:24.949252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1307/2597] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:02:24.961048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf614b-dd72-4e91-bea1-a61921d49bf6 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:24.971963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf614b-dd72-4e91-bea1-a61921d49bf6 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:24.977395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fc5f8-a9a5-4ce5-a38d-8fd4caecf5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:24.977717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fc5f8-a9a5-4ce5-a38d-8fd4caecf5f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:24.978104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fc5f8-a9a5-4ce5-a38d-8fd4caecf5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:24.978442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fc5f8-a9a5-4ce5-a38d-8fd4caecf5f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:24.979400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fc5f8-a9a5-4ce5-a38d-8fd4caecf5f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:24.981556 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fc5f8-a9a5-4ce5-a38d-8fd4caecf5f8 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:25.086320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:25.093553 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1291/2598] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:02:24 2022] POST /identity/v3/projects => generated 369 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:02:25.097845 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c23253-d58a-4f0f-a73c-a246089e4b3a admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:25.108240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c23253-d58a-4f0f-a73c-a246089e4b3a admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:25.171688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb2409-a9ea-4576-ad4c-bf5209af9044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:25.172399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb2409-a9ea-4576-ad4c-bf5209af9044 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:25.172779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb2409-a9ea-4576-ad4c-bf5209af9044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:25.173073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb2409-a9ea-4576-ad4c-bf5209af9044 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:25.173785 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb2409-a9ea-4576-ad4c-bf5209af9044 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:25.184075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb2409-a9ea-4576-ad4c-bf5209af9044 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:25.260408 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1308/2599] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:02:25 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:02:25.264952 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fc5f8-a9a5-4ce5-a38d-8fd4caecf5f8 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:25.275806 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fc5f8-a9a5-4ce5-a38d-8fd4caecf5f8 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:25.281037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff49b2-ad8c-46d2-8654-2502ef06ea6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:25.281784 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff49b2-ad8c-46d2-8654-2502ef06ea6c admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:25.282062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff49b2-ad8c-46d2-8654-2502ef06ea6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:25.282338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff49b2-ad8c-46d2-8654-2502ef06ea6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:25.283072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff49b2-ad8c-46d2-8654-2502ef06ea6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:25.284729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff49b2-ad8c-46d2-8654-2502ef06ea6c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:25.297018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1292/2600] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:25.302089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb2409-a9ea-4576-ad4c-bf5209af9044 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:25.314003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb2409-a9ea-4576-ad4c-bf5209af9044 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:25.318777 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449de84-842e-47f4-b55f-a67af24f6a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:25.319443 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449de84-842e-47f4-b55f-a67af24f6a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:25.319728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449de84-842e-47f4-b55f-a67af24f6a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:25.320009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449de84-842e-47f4-b55f-a67af24f6a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:25.320734 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449de84-842e-47f4-b55f-a67af24f6a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:25.322974 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449de84-842e-47f4-b55f-a67af24f6a3b admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:25.325356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:25.334609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1309/2601] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:25.339169 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff49b2-ad8c-46d2-8654-2502ef06ea6c admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:25.349320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff49b2-ad8c-46d2-8654-2502ef06ea6c admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:25.354535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c7267-2a39-4bf7-9708-487c607b0305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:25.355411 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c7267-2a39-4bf7-9708-487c607b0305 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:25.355694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c7267-2a39-4bf7-9708-487c607b0305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:25.355971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c7267-2a39-4bf7-9708-487c607b0305 admin admin] PATH_INFO: `/v3/projects/ee1dd6e32caa41cbbe41ee7212f3ed8b/users/eea21e1055a34f86bd522a1c58b7f939/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:25.372715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c7267-2a39-4bf7-9708-487c607b0305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee1dd6e32caa41cbbe41ee7212f3ed8b, user_id=eea21e1055a34f86bd522a1c58b7f939, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:25.375469 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c7267-2a39-4bf7-9708-487c607b0305 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:25.451500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1293/2602] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:25 2022] PUT /identity/v3/projects/ee1dd6e32caa41cbbe41ee7212f3ed8b/users/eea21e1055a34f86bd522a1c58b7f939/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:02:25.463047 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a1af1-b6c9-4572-91eb-1646f624e3f3 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:25.463541 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a1af1-b6c9-4572-91eb-1646f624e3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:25.463541 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a1af1-b6c9-4572-91eb-1646f624e3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:25.491431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-e11a1af1-b6c9-4572-91eb-1646f624e3f3 None None] MFA Rules not processed for user `eea21e1055a34f86bd522a1c58b7f939`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:25.492636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e11a1af1-b6c9-4572-91eb-1646f624e3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:25.570142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1310/2603] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:25.574832 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c7267-2a39-4bf7-9708-487c607b0305 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:25.638845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c7267-2a39-4bf7-9708-487c607b0305 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:25.643510 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339e7bc-7127-4a9a-a35c-1e6070546d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:25.644167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339e7bc-7127-4a9a-a35c-1e6070546d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:25.644447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339e7bc-7127-4a9a-a35c-1e6070546d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:25.644721 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339e7bc-7127-4a9a-a35c-1e6070546d3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:25.645435 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339e7bc-7127-4a9a-a35c-1e6070546d3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:25.647318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339e7bc-7127-4a9a-a35c-1e6070546d3a admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:25.746085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1294/2604] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:02:25 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:02:25.750085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a1af1-b6c9-4572-91eb-1646f624e3f3 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:25.761736 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a1af1-b6c9-4572-91eb-1646f624e3f3 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:25.822133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15a851-0887-4f7d-ba3e-bde2ebca099d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:25.822839 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15a851-0887-4f7d-ba3e-bde2ebca099d admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:25.823126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15a851-0887-4f7d-ba3e-bde2ebca099d admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:25.823402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15a851-0887-4f7d-ba3e-bde2ebca099d admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:25.824101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15a851-0887-4f7d-ba3e-bde2ebca099d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:25.826071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15a851-0887-4f7d-ba3e-bde2ebca099d admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:25.904409 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1311/2605] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:02:25 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:02:25.908152 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339e7bc-7127-4a9a-a35c-1e6070546d3a admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:25.920619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339e7bc-7127-4a9a-a35c-1e6070546d3a admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:25.925337 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e08b1a-8769-44f6-bcc9-41742e1f58af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:25.925993 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e08b1a-8769-44f6-bcc9-41742e1f58af admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:25.926276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e08b1a-8769-44f6-bcc9-41742e1f58af admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:25.926554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e08b1a-8769-44f6-bcc9-41742e1f58af admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:25.927292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e08b1a-8769-44f6-bcc9-41742e1f58af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:25.928885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e08b1a-8769-44f6-bcc9-41742e1f58af admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:25.940709 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1295/2606] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:02:25.944819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15a851-0887-4f7d-ba3e-bde2ebca099d admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:25.955044 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15a851-0887-4f7d-ba3e-bde2ebca099d admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:25.959659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f190d2-7940-47b8-b283-e77ca04ed07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:25.960474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f190d2-7940-47b8-b283-e77ca04ed07b admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:25.960754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f190d2-7940-47b8-b283-e77ca04ed07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:25.961068 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f190d2-7940-47b8-b283-e77ca04ed07b admin admin] PATH_INFO: `/v3/projects/11306cec6d9e47c59b03a64387ca51a6/users/1106f221ddd3485a90cf3fad329555de/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:25.976070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f190d2-7940-47b8-b283-e77ca04ed07b admin admin] RBAC: Authorizing `identity:create_grant(project_id=11306cec6d9e47c59b03a64387ca51a6, user_id=1106f221ddd3485a90cf3fad329555de, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:25.979378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f190d2-7940-47b8-b283-e77ca04ed07b admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:26.048234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1312/2607] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:25 2022] PUT /identity/v3/projects/11306cec6d9e47c59b03a64387ca51a6/users/1106f221ddd3485a90cf3fad329555de/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:26.052882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e08b1a-8769-44f6-bcc9-41742e1f58af admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:26.063706 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e08b1a-8769-44f6-bcc9-41742e1f58af admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:26.124581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24aa8bd-f79d-4d8f-9a2b-06b0a7475307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:26.125283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24aa8bd-f79d-4d8f-9a2b-06b0a7475307 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:26.125659 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24aa8bd-f79d-4d8f-9a2b-06b0a7475307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:26.125898 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24aa8bd-f79d-4d8f-9a2b-06b0a7475307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:26.126636 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24aa8bd-f79d-4d8f-9a2b-06b0a7475307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:26.128266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24aa8bd-f79d-4d8f-9a2b-06b0a7475307 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:26.139316 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1296/2608] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:26.143430 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f190d2-7940-47b8-b283-e77ca04ed07b admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:26.154239 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f190d2-7940-47b8-b283-e77ca04ed07b admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:26.159984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68515b10-168c-43aa-bed9-891451216096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:26.160583 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68515b10-168c-43aa-bed9-891451216096 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:26.160867 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68515b10-168c-43aa-bed9-891451216096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:26.161210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68515b10-168c-43aa-bed9-891451216096 admin admin] PATH_INFO: `/v3/projects/11306cec6d9e47c59b03a64387ca51a6/users/1106f221ddd3485a90cf3fad329555de/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:26.163419 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68515b10-168c-43aa-bed9-891451216096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11306cec6d9e47c59b03a64387ca51a6, user_id=1106f221ddd3485a90cf3fad329555de, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:26.166759 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68515b10-168c-43aa-bed9-891451216096 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:26.178974 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1313/2609] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:26 2022] PUT /identity/v3/projects/11306cec6d9e47c59b03a64387ca51a6/users/1106f221ddd3485a90cf3fad329555de/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:26.190729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a48bee-39d7-48d9-8975-1c61400a57ad None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:26.191062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a48bee-39d7-48d9-8975-1c61400a57ad None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:26.191349 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a48bee-39d7-48d9-8975-1c61400a57ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:26.219288 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-28a48bee-39d7-48d9-8975-1c61400a57ad None None] MFA Rules not processed for user `1106f221ddd3485a90cf3fad329555de`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:26.220579 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-28a48bee-39d7-48d9-8975-1c61400a57ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:26.246974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:26.293947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1297/2610] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:26.305372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68515b10-168c-43aa-bed9-891451216096 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:26.308465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-68515b10-168c-43aa-bed9-891451216096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:26.357089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:26.372657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68515b10-168c-43aa-bed9-891451216096 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:26.376434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519b0b2-e5ee-49bf-ae80-0bfbbcb5a807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:26.377125 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519b0b2-e5ee-49bf-ae80-0bfbbcb5a807 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:26.377414 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519b0b2-e5ee-49bf-ae80-0bfbbcb5a807 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:26.377692 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519b0b2-e5ee-49bf-ae80-0bfbbcb5a807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:26.387655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519b0b2-e5ee-49bf-ae80-0bfbbcb5a807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:26.389135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519b0b2-e5ee-49bf-ae80-0bfbbcb5a807 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:26.398970 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1314/2611] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:27.507769 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a48bee-39d7-48d9-8975-1c61400a57ad None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:27.575971 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a48bee-39d7-48d9-8975-1c61400a57ad None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:27.580516 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbad04-a7a0-453d-bc29-5d65f104c34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:27.581442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbad04-a7a0-453d-bc29-5d65f104c34e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:27.581723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbad04-a7a0-453d-bc29-5d65f104c34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:27.581998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbad04-a7a0-453d-bc29-5d65f104c34e admin admin] PATH_INFO: `/v3/users/eea21e1055a34f86bd522a1c58b7f939` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:27.583224 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbad04-a7a0-453d-bc29-5d65f104c34e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea21e1055a34f86bd522a1c58b7f939)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:27.585386 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbad04-a7a0-453d-bc29-5d65f104c34e admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:27.775146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-32dbad04-a7a0-453d-bc29-5d65f104c34e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea21e1055a34f86bd522a1c58b7f939'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:27.858701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-32dbad04-a7a0-453d-bc29-5d65f104c34e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea21e1055a34f86bd522a1c58b7f939'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:27.859351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-32dbad04-a7a0-453d-bc29-5d65f104c34e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea21e1055a34f86bd522a1c58b7f939'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:27.859956 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-32dbad04-a7a0-453d-bc29-5d65f104c34e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea21e1055a34f86bd522a1c58b7f939'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:27.873654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-32dbad04-a7a0-453d-bc29-5d65f104c34e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea21e1055a34f86bd522a1c58b7f939'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:27.886707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1298/2612] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:02:27 2022] DELETE /identity/v3/users/eea21e1055a34f86bd522a1c58b7f939 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:27.891500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489ce1a-7411-4368-a25b-c84e5d74bbea None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:27.892213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489ce1a-7411-4368-a25b-c84e5d74bbea None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:27.892213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c489ce1a-7411-4368-a25b-c84e5d74bbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:27.911617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-c489ce1a-7411-4368-a25b-c84e5d74bbea None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:27.913063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c489ce1a-7411-4368-a25b-c84e5d74bbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:27.984309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1315/2613] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:27.994006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbad04-a7a0-453d-bc29-5d65f104c34e admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:27.995523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-32dbad04-a7a0-453d-bc29-5d65f104c34e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:28.042204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:28.056713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbad04-a7a0-453d-bc29-5d65f104c34e admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:28.060654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755db308-9866-487f-882f-15b122159daa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:28.061520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755db308-9866-487f-882f-15b122159daa service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:28.061685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755db308-9866-487f-882f-15b122159daa service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:28.061904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755db308-9866-487f-882f-15b122159daa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:28.071344 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755db308-9866-487f-882f-15b122159daa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:28.073145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755db308-9866-487f-882f-15b122159daa service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:28.088322 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1299/2614] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:28.597434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489ce1a-7411-4368-a25b-c84e5d74bbea None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:28.598861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c489ce1a-7411-4368-a25b-c84e5d74bbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:28.613510 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c489ce1a-7411-4368-a25b-c84e5d74bbea None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:28.619315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:28.620230 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:28.620516 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:28.620793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde admin admin] PATH_INFO: `/v3/projects/ee1dd6e32caa41cbbe41ee7212f3ed8b` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:28.622128 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee1dd6e32caa41cbbe41ee7212f3ed8b)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:28.624101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:28.731320 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee1dd6e32caa41cbbe41ee7212f3ed8b', 'request_id': 'req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:28.742814 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee1dd6e32caa41cbbe41ee7212f3ed8b', 'request_id': 'req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:28.750599 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee1dd6e32caa41cbbe41ee7212f3ed8b', 'request_id': 'req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:28.852225 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde admin admin] The token cache is being invalidate because project ee1dd6e32caa41cbbe41ee7212f3ed8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:02:28.852428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:28.853779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:28.855439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1316/2615] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:02:28 2022] DELETE /identity/v3/projects/ee1dd6e32caa41cbbe41ee7212f3ed8b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:28.860052 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755db308-9866-487f-882f-15b122159daa service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:28.870846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755db308-9866-487f-882f-15b122159daa service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:28.945190 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bc3c3-28a7-4537-a1be-b1bab29c6a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:28.946162 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bc3c3-28a7-4537-a1be-b1bab29c6a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:28.946445 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bc3c3-28a7-4537-a1be-b1bab29c6a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:28.946726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bc3c3-28a7-4537-a1be-b1bab29c6a70 admin admin] PATH_INFO: `/v3/users/1106f221ddd3485a90cf3fad329555de` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:28.948518 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26bc3c3-28a7-4537-a1be-b1bab29c6a70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1106f221ddd3485a90cf3fad329555de)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:28.950755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26bc3c3-28a7-4537-a1be-b1bab29c6a70 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:29.207754 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-d26bc3c3-28a7-4537-a1be-b1bab29c6a70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1106f221ddd3485a90cf3fad329555de'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:29.296158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-d26bc3c3-28a7-4537-a1be-b1bab29c6a70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1106f221ddd3485a90cf3fad329555de'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:29.296782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-d26bc3c3-28a7-4537-a1be-b1bab29c6a70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1106f221ddd3485a90cf3fad329555de'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:29.297426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-d26bc3c3-28a7-4537-a1be-b1bab29c6a70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1106f221ddd3485a90cf3fad329555de'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:29.313268 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-d26bc3c3-28a7-4537-a1be-b1bab29c6a70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1106f221ddd3485a90cf3fad329555de'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:29.328269 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1300/2616] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:02:28 2022] DELETE /identity/v3/users/1106f221ddd3485a90cf3fad329555de => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:02:29.800628 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:29.802245 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:29.825784 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a4608-b473-46b7-a8ba-7c2f23e4bcde admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:29.901434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64601f-b64f-4aba-92ad-0e786aa56725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:29.902434 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64601f-b64f-4aba-92ad-0e786aa56725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:29.902721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64601f-b64f-4aba-92ad-0e786aa56725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:29.902997 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64601f-b64f-4aba-92ad-0e786aa56725 admin admin] PATH_INFO: `/v3/projects/11306cec6d9e47c59b03a64387ca51a6` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:29.904230 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64601f-b64f-4aba-92ad-0e786aa56725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11306cec6d9e47c59b03a64387ca51a6)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:29.906199 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64601f-b64f-4aba-92ad-0e786aa56725 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:29.916557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:30.025034 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0b64601f-b64f-4aba-92ad-0e786aa56725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11306cec6d9e47c59b03a64387ca51a6', 'request_id': 'req-0b64601f-b64f-4aba-92ad-0e786aa56725'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:30.031907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0b64601f-b64f-4aba-92ad-0e786aa56725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11306cec6d9e47c59b03a64387ca51a6', 'request_id': 'req-0b64601f-b64f-4aba-92ad-0e786aa56725'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:30.037632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0b64601f-b64f-4aba-92ad-0e786aa56725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11306cec6d9e47c59b03a64387ca51a6', 'request_id': 'req-0b64601f-b64f-4aba-92ad-0e786aa56725'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:30.146949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0b64601f-b64f-4aba-92ad-0e786aa56725 admin admin] The token cache is being invalidate because project 11306cec6d9e47c59b03a64387ca51a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:02:30.147356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0b64601f-b64f-4aba-92ad-0e786aa56725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:30.147976 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-0b64601f-b64f-4aba-92ad-0e786aa56725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:30.149589 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1317/2617] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:02:29 2022] DELETE /identity/v3/projects/11306cec6d9e47c59b03a64387ca51a6 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:30.155200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40d1b9-c125-4c67-8d05-a613dad1581d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:30.155666 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40d1b9-c125-4c67-8d05-a613dad1581d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:30.155749 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40d1b9-c125-4c67-8d05-a613dad1581d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:30.174889 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-da40d1b9-c125-4c67-8d05-a613dad1581d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:30.176084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-da40d1b9-c125-4c67-8d05-a613dad1581d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:30.237949 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1301/2618] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:30.245973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb694c1f-dd77-443b-a2d0-d915dd83a912 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:30.246261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb694c1f-dd77-443b-a2d0-d915dd83a912 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:30.246572 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb694c1f-dd77-443b-a2d0-d915dd83a912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:30.264702 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-cb694c1f-dd77-443b-a2d0-d915dd83a912 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:30.265941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-cb694c1f-dd77-443b-a2d0-d915dd83a912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:30.274708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1318/2619] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:30.279767 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40d1b9-c125-4c67-8d05-a613dad1581d None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:30.284586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:30.289918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40d1b9-c125-4c67-8d05-a613dad1581d None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:30.295339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b03682-04c1-4df3-bd48-0c0bb03be155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:30.296009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b03682-04c1-4df3-bd48-0c0bb03be155 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:30.296295 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b03682-04c1-4df3-bd48-0c0bb03be155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:30.296741 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b03682-04c1-4df3-bd48-0c0bb03be155 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:30.297535 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b03682-04c1-4df3-bd48-0c0bb03be155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:02:30.297846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b03682-04c1-4df3-bd48-0c0bb03be155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:30.299448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b03682-04c1-4df3-bd48-0c0bb03be155 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:30.312102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1302/2620] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:02:30.316594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb694c1f-dd77-443b-a2d0-d915dd83a912 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:30.326096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb694c1f-dd77-443b-a2d0-d915dd83a912 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:30.330720 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb13d3-a2b5-41be-925f-7b7a9ca29ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:30.331538 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb13d3-a2b5-41be-925f-7b7a9ca29ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:30.331841 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb13d3-a2b5-41be-925f-7b7a9ca29ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:30.332098 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb13d3-a2b5-41be-925f-7b7a9ca29ba7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:30.332782 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb13d3-a2b5-41be-925f-7b7a9ca29ba7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:30.334708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb13d3-a2b5-41be-925f-7b7a9ca29ba7 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:30.409749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1319/2621] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:02:30 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:02:30.414189 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b03682-04c1-4df3-bd48-0c0bb03be155 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:30.425250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b03682-04c1-4df3-bd48-0c0bb03be155 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:30.482215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c273592-00d9-4aee-be4c-059b638c25f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:30.482884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c273592-00d9-4aee-be4c-059b638c25f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:30.483164 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c273592-00d9-4aee-be4c-059b638c25f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:30.483726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c273592-00d9-4aee-be4c-059b638c25f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:30.484409 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c273592-00d9-4aee-be4c-059b638c25f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:30.486527 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c273592-00d9-4aee-be4c-059b638c25f5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:30.579758 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1303/2622] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:02:30 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:02:30.586097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb13d3-a2b5-41be-925f-7b7a9ca29ba7 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:30.597930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb13d3-a2b5-41be-925f-7b7a9ca29ba7 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:30.602759 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d4d68-e465-4983-859d-67e9ff3c866e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:30.603992 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d4d68-e465-4983-859d-67e9ff3c866e admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:30.604279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d4d68-e465-4983-859d-67e9ff3c866e admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:30.604556 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d4d68-e465-4983-859d-67e9ff3c866e admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:30.605333 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d4d68-e465-4983-859d-67e9ff3c866e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:30.606929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d4d68-e465-4983-859d-67e9ff3c866e admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:30.618679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1320/2623] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:02:30.622829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c273592-00d9-4aee-be4c-059b638c25f5 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:30.634826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c273592-00d9-4aee-be4c-059b638c25f5 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:30.643112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ed945-17f2-4477-8b30-52afe74d5f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:30.643796 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ed945-17f2-4477-8b30-52afe74d5f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:30.644100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ed945-17f2-4477-8b30-52afe74d5f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:30.644377 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ed945-17f2-4477-8b30-52afe74d5f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:30.645156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ed945-17f2-4477-8b30-52afe74d5f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:30.646754 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ed945-17f2-4477-8b30-52afe74d5f73 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:30.659843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1304/2624] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:02:30.663899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d4d68-e465-4983-859d-67e9ff3c866e admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:30.675166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d4d68-e465-4983-859d-67e9ff3c866e admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:30.679837 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63b025-d5db-4c50-8990-46e610b943bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:30.680792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63b025-d5db-4c50-8990-46e610b943bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:30.681205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63b025-d5db-4c50-8990-46e610b943bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:30.681717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63b025-d5db-4c50-8990-46e610b943bf admin admin] PATH_INFO: `/v3/projects/cb809ec74d0b471db8792c73a900112c/users/c1cf356bcc594f5987c24bf02f90bdcb/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:30.696718 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63b025-d5db-4c50-8990-46e610b943bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb809ec74d0b471db8792c73a900112c, user_id=c1cf356bcc594f5987c24bf02f90bdcb, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:30.703203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63b025-d5db-4c50-8990-46e610b943bf admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:30.802184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1321/2625] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:30 2022] PUT /identity/v3/projects/cb809ec74d0b471db8792c73a900112c/users/c1cf356bcc594f5987c24bf02f90bdcb/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:30.813446 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf6689-4aa8-43cb-93cd-23cdc4afa0f3 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:30.813737 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf6689-4aa8-43cb-93cd-23cdc4afa0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:30.814011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf6689-4aa8-43cb-93cd-23cdc4afa0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:30.841519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-3daf6689-4aa8-43cb-93cd-23cdc4afa0f3 None None] MFA Rules not processed for user `c1cf356bcc594f5987c24bf02f90bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:30.842725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3daf6689-4aa8-43cb-93cd-23cdc4afa0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:30.908566 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1305/2626] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:30.913176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63b025-d5db-4c50-8990-46e610b943bf admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:30.977646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63b025-d5db-4c50-8990-46e610b943bf admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:30.982889 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd86ee-fee2-4716-897b-88becde7f982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:30.983810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd86ee-fee2-4716-897b-88becde7f982 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:30.984117 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd86ee-fee2-4716-897b-88becde7f982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:30.984396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffd86ee-fee2-4716-897b-88becde7f982 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:30.985138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd86ee-fee2-4716-897b-88becde7f982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:30.987067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffd86ee-fee2-4716-897b-88becde7f982 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:31.069861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1322/2627] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:02:30 2022] POST /identity/v3/projects => generated 365 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:02:31.077748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf6689-4aa8-43cb-93cd-23cdc4afa0f3 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:31.089390 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf6689-4aa8-43cb-93cd-23cdc4afa0f3 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:31.148559 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:31.154624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4af46f-36c3-489a-89e2-20dc77364490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:31.155312 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4af46f-36c3-489a-89e2-20dc77364490 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:31.155834 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4af46f-36c3-489a-89e2-20dc77364490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:31.155936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4af46f-36c3-489a-89e2-20dc77364490 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:31.156569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4af46f-36c3-489a-89e2-20dc77364490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:31.164774 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4af46f-36c3-489a-89e2-20dc77364490 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:31.270841 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1306/2628] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:02:31 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:02:31.274841 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd86ee-fee2-4716-897b-88becde7f982 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:31.285578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffd86ee-fee2-4716-897b-88becde7f982 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:31.290297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37129470-0331-42a5-99b2-68a4c631bda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:31.291707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37129470-0331-42a5-99b2-68a4c631bda9 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:31.292034 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37129470-0331-42a5-99b2-68a4c631bda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:31.292324 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37129470-0331-42a5-99b2-68a4c631bda9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:31.293085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37129470-0331-42a5-99b2-68a4c631bda9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:31.294688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37129470-0331-42a5-99b2-68a4c631bda9 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:31.305356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1323/2629] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:31.310324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4af46f-36c3-489a-89e2-20dc77364490 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:31.320492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4af46f-36c3-489a-89e2-20dc77364490 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:31.326525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1ff9b-2904-4f63-8ab7-f6b8ad598107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:31.327430 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1ff9b-2904-4f63-8ab7-f6b8ad598107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:31.327713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1ff9b-2904-4f63-8ab7-f6b8ad598107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:31.327986 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1ff9b-2904-4f63-8ab7-f6b8ad598107 admin admin] PATH_INFO: `/v3/projects/293f4cc4516c4f2291e356183e19203b/users/e65396580fdd4c1783d3a4df871ec7fa/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:31.343308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1ff9b-2904-4f63-8ab7-f6b8ad598107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293f4cc4516c4f2291e356183e19203b, user_id=e65396580fdd4c1783d3a4df871ec7fa, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:31.345998 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1ff9b-2904-4f63-8ab7-f6b8ad598107 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:31.448720 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1307/2630] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:31 2022] PUT /identity/v3/projects/293f4cc4516c4f2291e356183e19203b/users/e65396580fdd4c1783d3a4df871ec7fa/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:31.452725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37129470-0331-42a5-99b2-68a4c631bda9 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:31.463273 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37129470-0331-42a5-99b2-68a4c631bda9 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:31.467138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:31.523310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e61eb2-5a78-4a27-95fb-9d53223adf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:31.523981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e61eb2-5a78-4a27-95fb-9d53223adf56 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:31.524267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e61eb2-5a78-4a27-95fb-9d53223adf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:31.524546 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e61eb2-5a78-4a27-95fb-9d53223adf56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:31.525483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e61eb2-5a78-4a27-95fb-9d53223adf56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:31.527195 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e61eb2-5a78-4a27-95fb-9d53223adf56 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:31.537964 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1324/2631] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:02:31.543010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1ff9b-2904-4f63-8ab7-f6b8ad598107 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:31.554052 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1ff9b-2904-4f63-8ab7-f6b8ad598107 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:31.560676 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6f565-9e4d-4198-a9f2-d63868da932a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:31.561544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6f565-9e4d-4198-a9f2-d63868da932a admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:31.561822 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6f565-9e4d-4198-a9f2-d63868da932a admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:31.562101 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6f565-9e4d-4198-a9f2-d63868da932a admin admin] PATH_INFO: `/v3/projects/293f4cc4516c4f2291e356183e19203b/users/e65396580fdd4c1783d3a4df871ec7fa/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:31.564304 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6f565-9e4d-4198-a9f2-d63868da932a admin admin] RBAC: Authorizing `identity:create_grant(project_id=293f4cc4516c4f2291e356183e19203b, user_id=e65396580fdd4c1783d3a4df871ec7fa, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:31.567159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6f565-9e4d-4198-a9f2-d63868da932a admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:31.578464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1308/2632] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:31 2022] PUT /identity/v3/projects/293f4cc4516c4f2291e356183e19203b/users/e65396580fdd4c1783d3a4df871ec7fa/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:31.589565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7f652-48ea-45a1-8680-d0f073e6ba5b None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:31.589858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7f652-48ea-45a1-8680-d0f073e6ba5b None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:31.590143 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7f652-48ea-45a1-8680-d0f073e6ba5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:31.617800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-18b7f652-48ea-45a1-8680-d0f073e6ba5b None None] MFA Rules not processed for user `e65396580fdd4c1783d3a4df871ec7fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:31.619123 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-18b7f652-48ea-45a1-8680-d0f073e6ba5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:31.689046 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1325/2633] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:31 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:31.700928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6f565-9e4d-4198-a9f2-d63868da932a admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:31.702713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-4ae6f565-9e4d-4198-a9f2-d63868da932a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:31.759324 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6f565-9e4d-4198-a9f2-d63868da932a admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:31.763616 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b3973-b7da-4a33-8bfb-6bf8ded962ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:31.764264 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b3973-b7da-4a33-8bfb-6bf8ded962ca service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:31.764552 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b3973-b7da-4a33-8bfb-6bf8ded962ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:31.764831 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b3973-b7da-4a33-8bfb-6bf8ded962ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:31.774247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b3973-b7da-4a33-8bfb-6bf8ded962ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:31.775825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42b3973-b7da-4a33-8bfb-6bf8ded962ca service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:31.785462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1309/2634] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:31 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:32.021130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7f652-48ea-45a1-8680-d0f073e6ba5b None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:32.031048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7f652-48ea-45a1-8680-d0f073e6ba5b None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:32.034628 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0044c56-ca27-4f32-af9a-50b12e60c6b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:32.035264 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0044c56-ca27-4f32-af9a-50b12e60c6b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:32.035544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0044c56-ca27-4f32-af9a-50b12e60c6b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:32.035822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0044c56-ca27-4f32-af9a-50b12e60c6b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:32.092439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0044c56-ca27-4f32-af9a-50b12e60c6b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:32.093908 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0044c56-ca27-4f32-af9a-50b12e60c6b9 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:32.102717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1326/2635] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:32 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:36.934993 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b3973-b7da-4a33-8bfb-6bf8ded962ca service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:36.936448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f42b3973-b7da-4a33-8bfb-6bf8ded962ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:37.028187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b3973-b7da-4a33-8bfb-6bf8ded962ca service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:37.038525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7774fd-0f24-463c-a63d-ce4898c6af84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:37.039515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7774fd-0f24-463c-a63d-ce4898c6af84 service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:37.039937 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7774fd-0f24-463c-a63d-ce4898c6af84 service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:37.040318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7774fd-0f24-463c-a63d-ce4898c6af84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:37.043825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3f7774fd-0f24-463c-a63d-ce4898c6af84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:37.128275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:37.143529 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7774fd-0f24-463c-a63d-ce4898c6af84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:37.145802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7774fd-0f24-463c-a63d-ce4898c6af84 service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:37.161670 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1310/2636] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:42.098733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=88 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:47.244689 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:48.744861 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0044c56-ca27-4f32-af9a-50b12e60c6b9 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:48.746307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-a0044c56-ca27-4f32-af9a-50b12e60c6b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:48.810916 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0044c56-ca27-4f32-af9a-50b12e60c6b9 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:48.814565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f3eb0-169e-46ea-8dc2-7ead0426d88e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:48.815242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f3eb0-169e-46ea-8dc2-7ead0426d88e service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:48.815528 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f3eb0-169e-46ea-8dc2-7ead0426d88e service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:48.818105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f3eb0-169e-46ea-8dc2-7ead0426d88e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:48.818105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-0e0f3eb0-169e-46ea-8dc2-7ead0426d88e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:48.905774 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f3eb0-169e-46ea-8dc2-7ead0426d88e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:48.907606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f3eb0-169e-46ea-8dc2-7ead0426d88e service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:48.918476 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1327/2637] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:50.950137 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7774fd-0f24-463c-a63d-ce4898c6af84 service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:50.961858 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7774fd-0f24-463c-a63d-ce4898c6af84 service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:50.967922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b766328-cf41-4993-8fba-c8e1fae417a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:50.968925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b766328-cf41-4993-8fba-c8e1fae417a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:50.969536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b766328-cf41-4993-8fba-c8e1fae417a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:50.970726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b766328-cf41-4993-8fba-c8e1fae417a9 admin admin] PATH_INFO: `/v3/users/c1cf356bcc594f5987c24bf02f90bdcb` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:50.972624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b766328-cf41-4993-8fba-c8e1fae417a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1cf356bcc594f5987c24bf02f90bdcb)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:50.974745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b766328-cf41-4993-8fba-c8e1fae417a9 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:51.230896 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7b766328-cf41-4993-8fba-c8e1fae417a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1cf356bcc594f5987c24bf02f90bdcb'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:51.322275 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7b766328-cf41-4993-8fba-c8e1fae417a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1cf356bcc594f5987c24bf02f90bdcb'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:51.322911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7b766328-cf41-4993-8fba-c8e1fae417a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1cf356bcc594f5987c24bf02f90bdcb'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:51.323540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7b766328-cf41-4993-8fba-c8e1fae417a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1cf356bcc594f5987c24bf02f90bdcb'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:51.337813 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-7b766328-cf41-4993-8fba-c8e1fae417a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1cf356bcc594f5987c24bf02f90bdcb'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:51.350980 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1311/2638] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:02:50 2022] DELETE /identity/v3/users/c1cf356bcc594f5987c24bf02f90bdcb => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:51.355665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9d4df-684f-4187-be8d-8d3442b11c20 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:51.359604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9d4df-684f-4187-be8d-8d3442b11c20 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:51.359604 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9d4df-684f-4187-be8d-8d3442b11c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:51.376482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-c3d9d4df-684f-4187-be8d-8d3442b11c20 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:51.377826 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c3d9d4df-684f-4187-be8d-8d3442b11c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:51.442465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1328/2639] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:51.455684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b766328-cf41-4993-8fba-c8e1fae417a9 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:51.457290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-7b766328-cf41-4993-8fba-c8e1fae417a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:51.558609 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b766328-cf41-4993-8fba-c8e1fae417a9 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:51.564780 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d82a09-5ab4-44d6-af94-25bc723b456e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:51.565576 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d82a09-5ab4-44d6-af94-25bc723b456e service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:51.566106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d82a09-5ab4-44d6-af94-25bc723b456e service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:51.566534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d82a09-5ab4-44d6-af94-25bc723b456e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:51.580719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d82a09-5ab4-44d6-af94-25bc723b456e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:51.580849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d82a09-5ab4-44d6-af94-25bc723b456e service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:51.593963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1312/2640] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:52.066464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9d4df-684f-4187-be8d-8d3442b11c20 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:52.070085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c3d9d4df-684f-4187-be8d-8d3442b11c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:52.112048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9d4df-684f-4187-be8d-8d3442b11c20 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:52.117399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a82392-2d78-4655-a3e0-8fc6678454d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:52.119772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a82392-2d78-4655-a3e0-8fc6678454d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:52.119772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a82392-2d78-4655-a3e0-8fc6678454d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:52.119772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a82392-2d78-4655-a3e0-8fc6678454d0 admin admin] PATH_INFO: `/v3/projects/cb809ec74d0b471db8792c73a900112c` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:52.120703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a82392-2d78-4655-a3e0-8fc6678454d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb809ec74d0b471db8792c73a900112c)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:52.122984 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a82392-2d78-4655-a3e0-8fc6678454d0 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:52.313920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:52.334967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-76a82392-2d78-4655-a3e0-8fc6678454d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb809ec74d0b471db8792c73a900112c', 'request_id': 'req-76a82392-2d78-4655-a3e0-8fc6678454d0'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:52.355206 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-76a82392-2d78-4655-a3e0-8fc6678454d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb809ec74d0b471db8792c73a900112c', 'request_id': 'req-76a82392-2d78-4655-a3e0-8fc6678454d0'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:52.383522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-76a82392-2d78-4655-a3e0-8fc6678454d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb809ec74d0b471db8792c73a900112c', 'request_id': 'req-76a82392-2d78-4655-a3e0-8fc6678454d0'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:52.512641 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-76a82392-2d78-4655-a3e0-8fc6678454d0 admin admin] The token cache is being invalidate because project cb809ec74d0b471db8792c73a900112c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:02:52.513124 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-76a82392-2d78-4655-a3e0-8fc6678454d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:52.516152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-76a82392-2d78-4655-a3e0-8fc6678454d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:52.516152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1329/2641] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:02:52 2022] DELETE /identity/v3/projects/cb809ec74d0b471db8792c73a900112c => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:52.520296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d82a09-5ab4-44d6-af94-25bc723b456e service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:52.531607 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d82a09-5ab4-44d6-af94-25bc723b456e service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:52.618884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf79fd-a73a-41b0-8c3a-f30fa56688b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:52.620594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf79fd-a73a-41b0-8c3a-f30fa56688b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:52.621241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf79fd-a73a-41b0-8c3a-f30fa56688b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:52.621804 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf79fd-a73a-41b0-8c3a-f30fa56688b1 admin admin] PATH_INFO: `/v3/users/e65396580fdd4c1783d3a4df871ec7fa` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:52.624316 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf79fd-a73a-41b0-8c3a-f30fa56688b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e65396580fdd4c1783d3a4df871ec7fa)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:52.628949 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf79fd-a73a-41b0-8c3a-f30fa56688b1 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:52.660472 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:52.877729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f9bf79fd-a73a-41b0-8c3a-f30fa56688b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e65396580fdd4c1783d3a4df871ec7fa'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:52.971337 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f9bf79fd-a73a-41b0-8c3a-f30fa56688b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e65396580fdd4c1783d3a4df871ec7fa'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:52.972448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f9bf79fd-a73a-41b0-8c3a-f30fa56688b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e65396580fdd4c1783d3a4df871ec7fa'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:52.973874 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f9bf79fd-a73a-41b0-8c3a-f30fa56688b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e65396580fdd4c1783d3a4df871ec7fa'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:52.991234 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f9bf79fd-a73a-41b0-8c3a-f30fa56688b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e65396580fdd4c1783d3a4df871ec7fa'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:53.005152 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1313/2642] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:02:52 2022] DELETE /identity/v3/users/e65396580fdd4c1783d3a4df871ec7fa => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:53.271400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a82392-2d78-4655-a3e0-8fc6678454d0 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:53.273580 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-76a82392-2d78-4655-a3e0-8fc6678454d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:53.292498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a82392-2d78-4655-a3e0-8fc6678454d0 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:53.373399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8b4b5-1a1e-4d23-a239-059e535334b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:53.373399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8b4b5-1a1e-4d23-a239-059e535334b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:53.373399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8b4b5-1a1e-4d23-a239-059e535334b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:53.373399 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8b4b5-1a1e-4d23-a239-059e535334b1 admin admin] PATH_INFO: `/v3/users/4beff9397579496c80fd6e93b7a81542` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:53.387693 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8b4b5-1a1e-4d23-a239-059e535334b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4beff9397579496c80fd6e93b7a81542)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:53.390161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8b4b5-1a1e-4d23-a239-059e535334b1 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:53.519183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf79fd-a73a-41b0-8c3a-f30fa56688b1 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:53.521523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f9bf79fd-a73a-41b0-8c3a-f30fa56688b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:53.542153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf79fd-a73a-41b0-8c3a-f30fa56688b1 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:53.550298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3103f2-a2d1-418d-8ad1-a102eeb0c467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:53.551530 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3103f2-a2d1-418d-8ad1-a102eeb0c467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:53.551954 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3103f2-a2d1-418d-8ad1-a102eeb0c467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:53.552380 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3103f2-a2d1-418d-8ad1-a102eeb0c467 admin admin] PATH_INFO: `/v3/projects/293f4cc4516c4f2291e356183e19203b` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:53.553889 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3103f2-a2d1-418d-8ad1-a102eeb0c467 admin admin] RBAC: Authorizing `identity:delete_project(project_id=293f4cc4516c4f2291e356183e19203b)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:53.555882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3103f2-a2d1-418d-8ad1-a102eeb0c467 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:53.663236 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f5a8b4b5-1a1e-4d23-a239-059e535334b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4beff9397579496c80fd6e93b7a81542'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:53.686445 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f5a8b4b5-1a1e-4d23-a239-059e535334b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4beff9397579496c80fd6e93b7a81542'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:53.696502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-ce3103f2-a2d1-418d-8ad1-a102eeb0c467 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293f4cc4516c4f2291e356183e19203b', 'request_id': 'req-ce3103f2-a2d1-418d-8ad1-a102eeb0c467'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:53.706070 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-ce3103f2-a2d1-418d-8ad1-a102eeb0c467 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293f4cc4516c4f2291e356183e19203b', 'request_id': 'req-ce3103f2-a2d1-418d-8ad1-a102eeb0c467'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:53.756369 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f5a8b4b5-1a1e-4d23-a239-059e535334b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4beff9397579496c80fd6e93b7a81542'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:53.768200 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f5a8b4b5-1a1e-4d23-a239-059e535334b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4beff9397579496c80fd6e93b7a81542'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:53.769249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f5a8b4b5-1a1e-4d23-a239-059e535334b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4beff9397579496c80fd6e93b7a81542'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:53.777213 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1330/2643] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:02:53 2022] DELETE /identity/v3/users/4beff9397579496c80fd6e93b7a81542 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:02:53.845369 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-ce3103f2-a2d1-418d-8ad1-a102eeb0c467 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293f4cc4516c4f2291e356183e19203b', 'request_id': 'req-ce3103f2-a2d1-418d-8ad1-a102eeb0c467'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:53.862283 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-ce3103f2-a2d1-418d-8ad1-a102eeb0c467 admin admin] The token cache is being invalidate because project 293f4cc4516c4f2291e356183e19203b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:02:53.862628 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-ce3103f2-a2d1-418d-8ad1-a102eeb0c467 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:53.863443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-ce3103f2-a2d1-418d-8ad1-a102eeb0c467 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:53.865692 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1314/2644] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:02:53 2022] DELETE /identity/v3/projects/293f4cc4516c4f2291e356183e19203b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:53.871097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375516e6-697d-48ac-831a-74865e5c1dd6 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:53.871460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375516e6-697d-48ac-831a-74865e5c1dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:53.871738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375516e6-697d-48ac-831a-74865e5c1dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:53.893477 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-375516e6-697d-48ac-831a-74865e5c1dd6 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:53.894823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-375516e6-697d-48ac-831a-74865e5c1dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:53.954126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:54.001096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1331/2645] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:54.006359 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7a2d9-53aa-4a11-a5ae-8e4d38201c94 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.006836 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7a2d9-53aa-4a11-a5ae-8e4d38201c94 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.007165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7a2d9-53aa-4a11-a5ae-8e4d38201c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.034416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-28c7a2d9-53aa-4a11-a5ae-8e4d38201c94 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:54.035842 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-28c7a2d9-53aa-4a11-a5ae-8e4d38201c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:54.044756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1315/2646] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:54.050728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375516e6-697d-48ac-831a-74865e5c1dd6 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:54.062345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375516e6-697d-48ac-831a-74865e5c1dd6 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:54.068057 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce0b2f-1e98-4e0a-9a77-41e7948ad0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:54.068844 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce0b2f-1e98-4e0a-9a77-41e7948ad0e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.069284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce0b2f-1e98-4e0a-9a77-41e7948ad0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.069606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce0b2f-1e98-4e0a-9a77-41e7948ad0e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.071756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce0b2f-1e98-4e0a-9a77-41e7948ad0e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:02:54.072157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce0b2f-1e98-4e0a-9a77-41e7948ad0e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:54.074051 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce0b2f-1e98-4e0a-9a77-41e7948ad0e5 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:54.088039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1332/2647] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:02:54.093643 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7a2d9-53aa-4a11-a5ae-8e4d38201c94 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:54.102930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce0b2f-1e98-4e0a-9a77-41e7948ad0e5 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:54.107391 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5bce0b2f-1e98-4e0a-9a77-41e7948ad0e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:54.109562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7a2d9-53aa-4a11-a5ae-8e4d38201c94 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:54.120672 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024ecd5-06c6-457a-a9b7-d7939d56f09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:54.130662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce0b2f-1e98-4e0a-9a77-41e7948ad0e5 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:54.138814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024ecd5-06c6-457a-a9b7-d7939d56f09c admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.139110 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8217c6-08ce-4147-86ab-b3db4f606dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:54.139502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024ecd5-06c6-457a-a9b7-d7939d56f09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.139788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024ecd5-06c6-457a-a9b7-d7939d56f09c admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.140484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8217c6-08ce-4147-86ab-b3db4f606dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.140949 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8217c6-08ce-4147-86ab-b3db4f606dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.141241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024ecd5-06c6-457a-a9b7-d7939d56f09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:54.141493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8217c6-08ce-4147-86ab-b3db4f606dcb admin admin] PATH_INFO: `/v3/projects/3ce59d9a516c4ec296e5f4e74627491b` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.143749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8217c6-08ce-4147-86ab-b3db4f606dcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ce59d9a516c4ec296e5f4e74627491b)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:54.145747 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024ecd5-06c6-457a-a9b7-d7939d56f09c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:54.147217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8217c6-08ce-4147-86ab-b3db4f606dcb admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:54.223476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1316/2648] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:02:54 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:02:54.230890 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024ecd5-06c6-457a-a9b7-d7939d56f09c admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:54.286381 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:54.286677 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ba8217c6-08ce-4147-86ab-b3db4f606dcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ce59d9a516c4ec296e5f4e74627491b', 'request_id': 'req-ba8217c6-08ce-4147-86ab-b3db4f606dcb'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:54.294539 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ba8217c6-08ce-4147-86ab-b3db4f606dcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ce59d9a516c4ec296e5f4e74627491b', 'request_id': 'req-ba8217c6-08ce-4147-86ab-b3db4f606dcb'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:54.300661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ba8217c6-08ce-4147-86ab-b3db4f606dcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ce59d9a516c4ec296e5f4e74627491b', 'request_id': 'req-ba8217c6-08ce-4147-86ab-b3db4f606dcb'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:54.304754 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024ecd5-06c6-457a-a9b7-d7939d56f09c admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:54.357243 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ba8217c6-08ce-4147-86ab-b3db4f606dcb admin admin] The token cache is being invalidate because project 3ce59d9a516c4ec296e5f4e74627491b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:02:54.357701 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ba8217c6-08ce-4147-86ab-b3db4f606dcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:54.358410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ba8217c6-08ce-4147-86ab-b3db4f606dcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:02:54.360295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1333/2649] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:02:54 2022] DELETE /identity/v3/projects/3ce59d9a516c4ec296e5f4e74627491b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:54.366748 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac0290-96bb-4358-b17f-47deeae4d8cb None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.367126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac0290-96bb-4358-b17f-47deeae4d8cb None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.367490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac0290-96bb-4358-b17f-47deeae4d8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.373654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13554a5e-672f-4e82-95ac-14a99856f530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:54.374351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13554a5e-672f-4e82-95ac-14a99856f530 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.374668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13554a5e-672f-4e82-95ac-14a99856f530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.374983 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13554a5e-672f-4e82-95ac-14a99856f530 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.375689 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13554a5e-672f-4e82-95ac-14a99856f530 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:54.377704 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13554a5e-672f-4e82-95ac-14a99856f530 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:54.386875 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-deac0290-96bb-4358-b17f-47deeae4d8cb None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:54.388577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-deac0290-96bb-4358-b17f-47deeae4d8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:54.397110 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1334/2650] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:54.405794 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897e679-c320-4b0e-87a6-2ef02e3b92da None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.406074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897e679-c320-4b0e-87a6-2ef02e3b92da None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.406359 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897e679-c320-4b0e-87a6-2ef02e3b92da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.426632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-5897e679-c320-4b0e-87a6-2ef02e3b92da None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:54.433549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5897e679-c320-4b0e-87a6-2ef02e3b92da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:54.437534 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1335/2651] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:54.443127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897e679-c320-4b0e-87a6-2ef02e3b92da None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:54.458135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897e679-c320-4b0e-87a6-2ef02e3b92da None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:54.467073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e32fe8-460a-481e-a30d-b902f56d5a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:54.468103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e32fe8-460a-481e-a30d-b902f56d5a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.468624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e32fe8-460a-481e-a30d-b902f56d5a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.469189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e32fe8-460a-481e-a30d-b902f56d5a8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.470506 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e32fe8-460a-481e-a30d-b902f56d5a8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:02:54.471089 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e32fe8-460a-481e-a30d-b902f56d5a8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:54.474340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e32fe8-460a-481e-a30d-b902f56d5a8f admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:54.487096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1317/2652] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:02:54 2022] POST /identity/v3/users => generated 283 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:02:54.488282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13554a5e-672f-4e82-95ac-14a99856f530 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:54.490090 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-13554a5e-672f-4e82-95ac-14a99856f530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:54.492952 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1336/2653] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 11:02:54.497878 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e32fe8-460a-481e-a30d-b902f56d5a8f admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:54.508048 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13554a5e-672f-4e82-95ac-14a99856f530 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:54.509833 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e32fe8-460a-481e-a30d-b902f56d5a8f admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:54.516528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1a7be-674d-4b3f-9c22-b7645cb398b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:54.517308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1a7be-674d-4b3f-9c22-b7645cb398b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.517594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1a7be-674d-4b3f-9c22-b7645cb398b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.522581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3591ce-f349-4f1b-ad0d-d93c25379fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:54.522745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1a7be-674d-4b3f-9c22-b7645cb398b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.523175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3591ce-f349-4f1b-ad0d-d93c25379fad admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.523279 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1a7be-674d-4b3f-9c22-b7645cb398b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:54.523394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3591ce-f349-4f1b-ad0d-d93c25379fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.523394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3591ce-f349-4f1b-ad0d-d93c25379fad admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.523394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3591ce-f349-4f1b-ad0d-d93c25379fad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:54.523635 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1a7be-674d-4b3f-9c22-b7645cb398b9 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:54.523715 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3591ce-f349-4f1b-ad0d-d93c25379fad admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:54.534869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1318/2654] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:54.540933 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1a7be-674d-4b3f-9c22-b7645cb398b9 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:54.556638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1a7be-674d-4b3f-9c22-b7645cb398b9 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:54.562523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c6f99-ccb1-4ca8-a28e-9610eb02e48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:54.563204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c6f99-ccb1-4ca8-a28e-9610eb02e48f admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.563484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c6f99-ccb1-4ca8-a28e-9610eb02e48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.563756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c6f99-ccb1-4ca8-a28e-9610eb02e48f admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.564490 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c6f99-ccb1-4ca8-a28e-9610eb02e48f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:54.566139 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4c6f99-ccb1-4ca8-a28e-9610eb02e48f admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:54.579522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1319/2655] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:54.584114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c6f99-ccb1-4ca8-a28e-9610eb02e48f admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:54.594539 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c6f99-ccb1-4ca8-a28e-9610eb02e48f admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:54.599529 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c628c6-ae05-480e-93bb-a48bff0f697f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:54.600560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c628c6-ae05-480e-93bb-a48bff0f697f admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.600802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c628c6-ae05-480e-93bb-a48bff0f697f admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.601127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c628c6-ae05-480e-93bb-a48bff0f697f admin admin] PATH_INFO: `/v3/projects/1efc1eb4210047d6a0718d3f13eae9d5/users/4e22ac2d746447f198dd23f539646da2/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.618109 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c628c6-ae05-480e-93bb-a48bff0f697f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1efc1eb4210047d6a0718d3f13eae9d5, user_id=4e22ac2d746447f198dd23f539646da2, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:54.620817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c628c6-ae05-480e-93bb-a48bff0f697f admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:54.655907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1337/2656] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:02:54 2022] POST /identity/v3/projects => generated 405 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:02:54.660658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3591ce-f349-4f1b-ad0d-d93c25379fad admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:54.733285 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3591ce-f349-4f1b-ad0d-d93c25379fad admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:54.740396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97350df2-e772-442c-81c0-59bb2f4e5396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:54.741170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97350df2-e772-442c-81c0-59bb2f4e5396 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.741455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97350df2-e772-442c-81c0-59bb2f4e5396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.746097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97350df2-e772-442c-81c0-59bb2f4e5396 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.746917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97350df2-e772-442c-81c0-59bb2f4e5396 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:54.749806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97350df2-e772-442c-81c0-59bb2f4e5396 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:54.759359 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:54.774353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1320/2657] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:54 2022] PUT /identity/v3/projects/1efc1eb4210047d6a0718d3f13eae9d5/users/4e22ac2d746447f198dd23f539646da2/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:02:54.785539 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa52f4-1e1c-40b1-bef1-5e773248866c None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.785932 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa52f4-1e1c-40b1-bef1-5e773248866c None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.786107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa52f4-1e1c-40b1-bef1-5e773248866c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.815528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-64fa52f4-1e1c-40b1-bef1-5e773248866c None None] MFA Rules not processed for user `4e22ac2d746447f198dd23f539646da2`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:54.819094 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-64fa52f4-1e1c-40b1-bef1-5e773248866c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:54.860985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1338/2658] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:02:54 2022] POST /identity/v3/users => generated 303 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:02:54.865166 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97350df2-e772-442c-81c0-59bb2f4e5396 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:54.876322 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97350df2-e772-442c-81c0-59bb2f4e5396 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:54.887937 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1321/2659] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:54.890965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa52f4-1e1c-40b1-bef1-5e773248866c None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:54.945178 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdc287-4316-471d-b661-656b21e668cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:54.945907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdc287-4316-471d-b661-656b21e668cb admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.946164 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdc287-4316-471d-b661-656b21e668cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.946435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdc287-4316-471d-b661-656b21e668cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.947216 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdc287-4316-471d-b661-656b21e668cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:54.948911 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdc287-4316-471d-b661-656b21e668cb admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:54.961081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa52f4-1e1c-40b1-bef1-5e773248866c None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:54.961544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1339/2660] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:02:54.965498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdc287-4316-471d-b661-656b21e668cb admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:54.967094 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b1cf8-fb8b-4ff9-8074-a6c01df7ab26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:54.968590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b1cf8-fb8b-4ff9-8074-a6c01df7ab26 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.969118 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b1cf8-fb8b-4ff9-8074-a6c01df7ab26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.969602 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b1cf8-fb8b-4ff9-8074-a6c01df7ab26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.970836 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b1cf8-fb8b-4ff9-8074-a6c01df7ab26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:54.976426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b1cf8-fb8b-4ff9-8074-a6c01df7ab26 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:54.978228 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdc287-4316-471d-b661-656b21e668cb admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:54.985648 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad089588-ad8b-45c2-bc92-c220223e5c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:54.986947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad089588-ad8b-45c2-bc92-c220223e5c5d admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:54.987417 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad089588-ad8b-45c2-bc92-c220223e5c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:54.987817 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad089588-ad8b-45c2-bc92-c220223e5c5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:54.989972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad089588-ad8b-45c2-bc92-c220223e5c5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:54.992376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad089588-ad8b-45c2-bc92-c220223e5c5d admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:55.020884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1340/2661] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:02:55.023205 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad089588-ad8b-45c2-bc92-c220223e5c5d admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:55.035177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:55.036270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad089588-ad8b-45c2-bc92-c220223e5c5d admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:55.041466 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e61f8b-54f6-4d0c-b759-9a0000af1fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:55.042428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e61f8b-54f6-4d0c-b759-9a0000af1fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:55.042712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e61f8b-54f6-4d0c-b759-9a0000af1fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:55.042992 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e61f8b-54f6-4d0c-b759-9a0000af1fcf admin admin] PATH_INFO: `/v3/projects/d72138ef91eb4a9d8d5d1862fcfa7589/users/cf6233f345f84fc1a8458df668afc3f7/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:55.064655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e61f8b-54f6-4d0c-b759-9a0000af1fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72138ef91eb4a9d8d5d1862fcfa7589, user_id=cf6233f345f84fc1a8458df668afc3f7, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:55.069798 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e61f8b-54f6-4d0c-b759-9a0000af1fcf admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:55.107981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1322/2662] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:02:54 2022] POST /identity/v3/projects => generated 361 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:02:55.112811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b1cf8-fb8b-4ff9-8074-a6c01df7ab26 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:55.195821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b1cf8-fb8b-4ff9-8074-a6c01df7ab26 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:55.205152 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ecdcd-18c3-489d-9028-62af94ea5000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:55.205843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ecdcd-18c3-489d-9028-62af94ea5000 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:55.206130 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ecdcd-18c3-489d-9028-62af94ea5000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:55.206411 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ecdcd-18c3-489d-9028-62af94ea5000 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:55.207446 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ecdcd-18c3-489d-9028-62af94ea5000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:55.207613 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1341/2663] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:55 2022] PUT /identity/v3/projects/d72138ef91eb4a9d8d5d1862fcfa7589/users/cf6233f345f84fc1a8458df668afc3f7/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:55.209090 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ecdcd-18c3-489d-9028-62af94ea5000 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:55.212913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7c6a9-ec7f-4c92-8196-2dd38355292f None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:55.213217 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7c6a9-ec7f-4c92-8196-2dd38355292f None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:55.213498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7c6a9-ec7f-4c92-8196-2dd38355292f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:55.233025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-25d7c6a9-ec7f-4c92-8196-2dd38355292f None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:55.234154 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-25d7c6a9-ec7f-4c92-8196-2dd38355292f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:55.301725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1342/2664] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:02:55.304865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1323/2665] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:02:55 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:02:55.308803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7c6a9-ec7f-4c92-8196-2dd38355292f None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:55.312506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ecdcd-18c3-489d-9028-62af94ea5000 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:55.315448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-5b5ecdcd-18c3-489d-9028-62af94ea5000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:55.322501 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7c6a9-ec7f-4c92-8196-2dd38355292f None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:55.328265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832a3d6-178b-41a2-ab79-d6151b6b58f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:55.329267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832a3d6-178b-41a2-ab79-d6151b6b58f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:55.329532 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832a3d6-178b-41a2-ab79-d6151b6b58f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:55.329815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832a3d6-178b-41a2-ab79-d6151b6b58f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:55.330606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832a3d6-178b-41a2-ab79-d6151b6b58f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:55.332201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832a3d6-178b-41a2-ab79-d6151b6b58f5 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:55.345784 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1343/2666] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:55.351789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832a3d6-178b-41a2-ab79-d6151b6b58f5 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:55.364910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832a3d6-178b-41a2-ab79-d6151b6b58f5 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:55.370057 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe4583-c5c0-413f-a92e-5d7bf8b00f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:55.370948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe4583-c5c0-413f-a92e-5d7bf8b00f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:55.371233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe4583-c5c0-413f-a92e-5d7bf8b00f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:55.371501 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe4583-c5c0-413f-a92e-5d7bf8b00f76 admin admin] PATH_INFO: `/v3/projects/b35b1794fc0443af8f0733114d417d5a/users/8d06237344b74073a9e3ade002389744/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:55.391188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe4583-c5c0-413f-a92e-5d7bf8b00f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35b1794fc0443af8f0733114d417d5a, user_id=8d06237344b74073a9e3ade002389744, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:55.396518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe4583-c5c0-413f-a92e-5d7bf8b00f76 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:55.403163 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ecdcd-18c3-489d-9028-62af94ea5000 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:55.461921 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da6a72-9caa-41fb-aa6d-bb506e19fd4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:55.462600 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da6a72-9caa-41fb-aa6d-bb506e19fd4e service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:55.462882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da6a72-9caa-41fb-aa6d-bb506e19fd4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:55.463157 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da6a72-9caa-41fb-aa6d-bb506e19fd4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:55.473250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da6a72-9caa-41fb-aa6d-bb506e19fd4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:55.474675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da6a72-9caa-41fb-aa6d-bb506e19fd4e service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:55.489382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1324/2667] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:55.519994 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1344/2668] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:55 2022] PUT /identity/v3/projects/b35b1794fc0443af8f0733114d417d5a/users/8d06237344b74073a9e3ade002389744/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:02:55.524194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da6a72-9caa-41fb-aa6d-bb506e19fd4e service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:55.529665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:55.535483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da6a72-9caa-41fb-aa6d-bb506e19fd4e service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:55.601885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ca092-e92d-4034-a8c8-81fc72cb92f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:55.602578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ca092-e92d-4034-a8c8-81fc72cb92f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:55.602869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ca092-e92d-4034-a8c8-81fc72cb92f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:55.603148 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ca092-e92d-4034-a8c8-81fc72cb92f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:55.603900 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ca092-e92d-4034-a8c8-81fc72cb92f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:55.605801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ca092-e92d-4034-a8c8-81fc72cb92f6 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:55.618122 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1325/2669] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:02:55.623351 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe4583-c5c0-413f-a92e-5d7bf8b00f76 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:55.637394 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe4583-c5c0-413f-a92e-5d7bf8b00f76 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:55.646640 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59851e70-0bb0-437a-a0ff-e79ef6bca791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:55.648273 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59851e70-0bb0-437a-a0ff-e79ef6bca791 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:55.648807 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59851e70-0bb0-437a-a0ff-e79ef6bca791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:55.649563 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59851e70-0bb0-437a-a0ff-e79ef6bca791 admin admin] PATH_INFO: `/v3/projects/b35b1794fc0443af8f0733114d417d5a/users/8d06237344b74073a9e3ade002389744/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:55.652990 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59851e70-0bb0-437a-a0ff-e79ef6bca791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35b1794fc0443af8f0733114d417d5a, user_id=8d06237344b74073a9e3ade002389744, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:55.658870 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59851e70-0bb0-437a-a0ff-e79ef6bca791 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:55.676071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1345/2670] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:55 2022] PUT /identity/v3/projects/b35b1794fc0443af8f0733114d417d5a/users/8d06237344b74073a9e3ade002389744/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:02:55.687293 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6c867-b6df-4c8b-9c97-1367f6d428a9 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:55.687599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6c867-b6df-4c8b-9c97-1367f6d428a9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:55.687901 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6c867-b6df-4c8b-9c97-1367f6d428a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:55.721243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-cfb6c867-b6df-4c8b-9c97-1367f6d428a9 None None] MFA Rules not processed for user `8d06237344b74073a9e3ade002389744`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:55.722522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-cfb6c867-b6df-4c8b-9c97-1367f6d428a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:55.803079 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1326/2671] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:55 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:55.819041 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59851e70-0bb0-437a-a0ff-e79ef6bca791 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:55.822107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-59851e70-0bb0-437a-a0ff-e79ef6bca791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:55.835221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:55.898670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59851e70-0bb0-437a-a0ff-e79ef6bca791 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:55.905704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9e2d9-f7e3-4e5a-a39d-1b5ad669e6b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:55.906507 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9e2d9-f7e3-4e5a-a39d-1b5ad669e6b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:55.906880 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9e2d9-f7e3-4e5a-a39d-1b5ad669e6b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:55.907356 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9e2d9-f7e3-4e5a-a39d-1b5ad669e6b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:55.924605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9e2d9-f7e3-4e5a-a39d-1b5ad669e6b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:55.926764 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9e2d9-f7e3-4e5a-a39d-1b5ad669e6b8 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:55.944691 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1346/2672] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:55 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:02:56.161219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6c867-b6df-4c8b-9c97-1367f6d428a9 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:56.165142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-cfb6c867-b6df-4c8b-9c97-1367f6d428a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:56.178773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6c867-b6df-4c8b-9c97-1367f6d428a9 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:56.183251 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c90a9-d097-484a-a27f-0913284c09f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:56.184162 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c90a9-d097-484a-a27f-0913284c09f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:56.184606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c90a9-d097-484a-a27f-0913284c09f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:56.189334 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c90a9-d097-484a-a27f-0913284c09f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:56.277856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c90a9-d097-484a-a27f-0913284c09f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:56.281494 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c90a9-d097-484a-a27f-0913284c09f2 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:56.301575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1327/2673] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:02:56 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 11:02:56.892980 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bd054-6011-4961-ad07-15dcb6eace22 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:56.893549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bd054-6011-4961-ad07-15dcb6eace22 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:56.893756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bd054-6011-4961-ad07-15dcb6eace22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:56.927783 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-fc3bd054-6011-4961-ad07-15dcb6eace22 None None] MFA Rules not processed for user `cf6233f345f84fc1a8458df668afc3f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:56.929064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-fc3bd054-6011-4961-ad07-15dcb6eace22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:57.024225 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1347/2674] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:56 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:02:57.032318 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c90a9-d097-484a-a27f-0913284c09f2 service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:57.055839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:57.096177 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c90a9-d097-484a-a27f-0913284c09f2 service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:57.101754 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d4c23-6a36-411c-b30a-cd92e7d312d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:57.102634 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d4c23-6a36-411c-b30a-cd92e7d312d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:57.102980 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d4c23-6a36-411c-b30a-cd92e7d312d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:57.103261 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d4c23-6a36-411c-b30a-cd92e7d312d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:57.104339 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d4c23-6a36-411c-b30a-cd92e7d312d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:57.107548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d4c23-6a36-411c-b30a-cd92e7d312d6 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:57.207298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1328/2675] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:02:57 2022] POST /identity/v3/projects => generated 405 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:02:57.211554 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bd054-6011-4961-ad07-15dcb6eace22 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:57.223062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bd054-6011-4961-ad07-15dcb6eace22 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:57.287175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88074f8f-6044-4c8e-85b9-837b421f19aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:57.287862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88074f8f-6044-4c8e-85b9-837b421f19aa admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:57.288147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88074f8f-6044-4c8e-85b9-837b421f19aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:57.288430 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88074f8f-6044-4c8e-85b9-837b421f19aa admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:57.289151 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88074f8f-6044-4c8e-85b9-837b421f19aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:57.291108 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88074f8f-6044-4c8e-85b9-837b421f19aa admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:57.409441 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1348/2676] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:02:57 2022] POST /identity/v3/users => generated 309 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:02:57.415583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d4c23-6a36-411c-b30a-cd92e7d312d6 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:57.426886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d4c23-6a36-411c-b30a-cd92e7d312d6 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:57.431430 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417a64f-9b66-43ab-9e6b-5fc019fbc6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:57.432087 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417a64f-9b66-43ab-9e6b-5fc019fbc6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:57.432395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417a64f-9b66-43ab-9e6b-5fc019fbc6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:57.432650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417a64f-9b66-43ab-9e6b-5fc019fbc6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:57.433425 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417a64f-9b66-43ab-9e6b-5fc019fbc6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:57.435019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4417a64f-9b66-43ab-9e6b-5fc019fbc6e9 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:57.448829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1329/2677] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:57.454091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88074f8f-6044-4c8e-85b9-837b421f19aa admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:57.464801 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88074f8f-6044-4c8e-85b9-837b421f19aa admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:57.470423 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bfb54-1410-4ce9-990b-033367d795df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:57.471340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bfb54-1410-4ce9-990b-033367d795df admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:57.471667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bfb54-1410-4ce9-990b-033367d795df admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:57.471968 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bfb54-1410-4ce9-990b-033367d795df admin admin] PATH_INFO: `/v3/projects/1e1ba892d93d4a9f8cddd5489f7334ac/users/b1614bbbef9d457a9a298235fe8bca7b/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:57.492241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bfb54-1410-4ce9-990b-033367d795df admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e1ba892d93d4a9f8cddd5489f7334ac, user_id=b1614bbbef9d457a9a298235fe8bca7b, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:57.495337 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bfb54-1410-4ce9-990b-033367d795df admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:57.603380 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1349/2678] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:57 2022] PUT /identity/v3/projects/1e1ba892d93d4a9f8cddd5489f7334ac/users/b1614bbbef9d457a9a298235fe8bca7b/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:02:57.607351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417a64f-9b66-43ab-9e6b-5fc019fbc6e9 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:57.619410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417a64f-9b66-43ab-9e6b-5fc019fbc6e9 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:57.682241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d1212-f44f-4e9a-b129-1e59dd9d8635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:57.682920 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d1212-f44f-4e9a-b129-1e59dd9d8635 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:57.683208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d1212-f44f-4e9a-b129-1e59dd9d8635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:57.683488 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d1212-f44f-4e9a-b129-1e59dd9d8635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:57.684232 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d1212-f44f-4e9a-b129-1e59dd9d8635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:57.685854 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d1212-f44f-4e9a-b129-1e59dd9d8635 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:57.702965 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1330/2679] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:02:57.708146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bfb54-1410-4ce9-990b-033367d795df admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:02:57.714913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:02:57.719667 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bfb54-1410-4ce9-990b-033367d795df admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:02:57.726185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315887b0-18e4-4202-a61b-1fc7fd0dde0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:02:57.727841 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315887b0-18e4-4202-a61b-1fc7fd0dde0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:57.728331 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315887b0-18e4-4202-a61b-1fc7fd0dde0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:57.728874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315887b0-18e4-4202-a61b-1fc7fd0dde0e admin admin] PATH_INFO: `/v3/projects/1e1ba892d93d4a9f8cddd5489f7334ac/users/b1614bbbef9d457a9a298235fe8bca7b/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:57.732091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315887b0-18e4-4202-a61b-1fc7fd0dde0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e1ba892d93d4a9f8cddd5489f7334ac, user_id=b1614bbbef9d457a9a298235fe8bca7b, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:02:57.736439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315887b0-18e4-4202-a61b-1fc7fd0dde0e admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:02:57.748052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1350/2680] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:02:57 2022] PUT /identity/v3/projects/1e1ba892d93d4a9f8cddd5489f7334ac/users/b1614bbbef9d457a9a298235fe8bca7b/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:02:59.268395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa8d4b-0f52-4aba-a74f-0a35bfeaaab6 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:02:59.270195 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa8d4b-0f52-4aba-a74f-0a35bfeaaab6 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:02:59.270715 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa8d4b-0f52-4aba-a74f-0a35bfeaaab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:02:59.327189 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-7ffa8d4b-0f52-4aba-a74f-0a35bfeaaab6 None None] MFA Rules not processed for user `b1614bbbef9d457a9a298235fe8bca7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:02:59.329815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-7ffa8d4b-0f52-4aba-a74f-0a35bfeaaab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:02:59.454924 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1331/2681] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:02:59 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:03:00.085595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315887b0-18e4-4202-a61b-1fc7fd0dde0e admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:00.163489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315887b0-18e4-4202-a61b-1fc7fd0dde0e admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:00.168429 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51bbf3-4e44-4959-b50f-bc55e77ee5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:00.169369 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51bbf3-4e44-4959-b50f-bc55e77ee5db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:00.170008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51bbf3-4e44-4959-b50f-bc55e77ee5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:00.170008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51bbf3-4e44-4959-b50f-bc55e77ee5db admin admin] PATH_INFO: `/v3/users/4e22ac2d746447f198dd23f539646da2` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:00.171182 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51bbf3-4e44-4959-b50f-bc55e77ee5db admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e22ac2d746447f198dd23f539646da2)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:00.173326 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51bbf3-4e44-4959-b50f-bc55e77ee5db admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:00.439544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ab51bbf3-4e44-4959-b50f-bc55e77ee5db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e22ac2d746447f198dd23f539646da2'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:00.452083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ab51bbf3-4e44-4959-b50f-bc55e77ee5db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e22ac2d746447f198dd23f539646da2'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:00.528170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ab51bbf3-4e44-4959-b50f-bc55e77ee5db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e22ac2d746447f198dd23f539646da2'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:00.537800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ab51bbf3-4e44-4959-b50f-bc55e77ee5db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e22ac2d746447f198dd23f539646da2'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:00.538543 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ab51bbf3-4e44-4959-b50f-bc55e77ee5db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e22ac2d746447f198dd23f539646da2'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:00.545885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1351/2682] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:00 2022] DELETE /identity/v3/users/4e22ac2d746447f198dd23f539646da2 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:00.551475 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1e83e-83a6-4696-88f8-98b68da79411 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:00.551918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1e83e-83a6-4696-88f8-98b68da79411 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:00.552025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff1e83e-83a6-4696-88f8-98b68da79411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:00.570765 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-eff1e83e-83a6-4696-88f8-98b68da79411 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:00.572070 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-eff1e83e-83a6-4696-88f8-98b68da79411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:00.585337 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:00.641014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1332/2683] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:00.651112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51bbf3-4e44-4959-b50f-bc55e77ee5db admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:00.652562 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ab51bbf3-4e44-4959-b50f-bc55e77ee5db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:00.716996 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51bbf3-4e44-4959-b50f-bc55e77ee5db admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:00.721944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce85722-d1b7-47b7-a58d-cf8faae2d87e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:00.722611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce85722-d1b7-47b7-a58d-cf8faae2d87e service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:00.722898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce85722-d1b7-47b7-a58d-cf8faae2d87e service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:00.723206 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce85722-d1b7-47b7-a58d-cf8faae2d87e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:00.733305 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce85722-d1b7-47b7-a58d-cf8faae2d87e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:00.735653 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce85722-d1b7-47b7-a58d-cf8faae2d87e service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:00.749491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1352/2684] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:00.976836 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1e83e-83a6-4696-88f8-98b68da79411 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:00.978430 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-eff1e83e-83a6-4696-88f8-98b68da79411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:00.997732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff1e83e-83a6-4696-88f8-98b68da79411 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:01.004393 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e999da-3f5b-4481-a4fd-5527e5c4aa55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:01.005273 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e999da-3f5b-4481-a4fd-5527e5c4aa55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:01.005583 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e999da-3f5b-4481-a4fd-5527e5c4aa55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:01.005861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e999da-3f5b-4481-a4fd-5527e5c4aa55 admin admin] PATH_INFO: `/v3/users/cf6233f345f84fc1a8458df668afc3f7` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:01.007192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e999da-3f5b-4481-a4fd-5527e5c4aa55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf6233f345f84fc1a8458df668afc3f7)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:01.009165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e999da-3f5b-4481-a4fd-5527e5c4aa55 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:01.221342 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f5e999da-3f5b-4481-a4fd-5527e5c4aa55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf6233f345f84fc1a8458df668afc3f7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:01.244193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce85722-d1b7-47b7-a58d-cf8faae2d87e service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:01.245675 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-4ce85722-d1b7-47b7-a58d-cf8faae2d87e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:01.260878 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce85722-d1b7-47b7-a58d-cf8faae2d87e service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:01.266858 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853bc57-92c9-4a02-8919-d601c360ba3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:01.267838 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853bc57-92c9-4a02-8919-d601c360ba3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:01.268132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853bc57-92c9-4a02-8919-d601c360ba3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:01.268404 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853bc57-92c9-4a02-8919-d601c360ba3a admin admin] PATH_INFO: `/v3/projects/1efc1eb4210047d6a0718d3f13eae9d5` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:01.269967 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853bc57-92c9-4a02-8919-d601c360ba3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1efc1eb4210047d6a0718d3f13eae9d5)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:01.272486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853bc57-92c9-4a02-8919-d601c360ba3a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:01.282359 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:01.314429 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f5e999da-3f5b-4481-a4fd-5527e5c4aa55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf6233f345f84fc1a8458df668afc3f7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:01.315052 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f5e999da-3f5b-4481-a4fd-5527e5c4aa55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf6233f345f84fc1a8458df668afc3f7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:01.315723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f5e999da-3f5b-4481-a4fd-5527e5c4aa55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf6233f345f84fc1a8458df668afc3f7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:01.329484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f5e999da-3f5b-4481-a4fd-5527e5c4aa55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf6233f345f84fc1a8458df668afc3f7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:01.341712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1333/2685] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:00 2022] DELETE /identity/v3/users/cf6233f345f84fc1a8458df668afc3f7 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:01.447962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1853bc57-92c9-4a02-8919-d601c360ba3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1efc1eb4210047d6a0718d3f13eae9d5', 'request_id': 'req-1853bc57-92c9-4a02-8919-d601c360ba3a'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:01.455986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1853bc57-92c9-4a02-8919-d601c360ba3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1efc1eb4210047d6a0718d3f13eae9d5', 'request_id': 'req-1853bc57-92c9-4a02-8919-d601c360ba3a'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:01.461712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1853bc57-92c9-4a02-8919-d601c360ba3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1efc1eb4210047d6a0718d3f13eae9d5', 'request_id': 'req-1853bc57-92c9-4a02-8919-d601c360ba3a'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:01.559301 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1853bc57-92c9-4a02-8919-d601c360ba3a admin admin] The token cache is being invalidate because project 1efc1eb4210047d6a0718d3f13eae9d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:01.559590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1853bc57-92c9-4a02-8919-d601c360ba3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:01.563801 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1853bc57-92c9-4a02-8919-d601c360ba3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:01.563801 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1353/2686] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:01 2022] DELETE /identity/v3/projects/1efc1eb4210047d6a0718d3f13eae9d5 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:01.567468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e999da-3f5b-4481-a4fd-5527e5c4aa55 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:01.568892 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f5e999da-3f5b-4481-a4fd-5527e5c4aa55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:01.638295 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e999da-3f5b-4481-a4fd-5527e5c4aa55 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:01.672211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:01.691003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853bc57-92c9-4a02-8919-d601c360ba3a admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:01.692549 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-1853bc57-92c9-4a02-8919-d601c360ba3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:01.700254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f3de5-f730-444d-93cd-e4fd7e23223f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:01.701221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f3de5-f730-444d-93cd-e4fd7e23223f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:01.701505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f3de5-f730-444d-93cd-e4fd7e23223f admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:01.701788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f3de5-f730-444d-93cd-e4fd7e23223f admin admin] PATH_INFO: `/v3/users/8d06237344b74073a9e3ade002389744` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:01.703153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f3de5-f730-444d-93cd-e4fd7e23223f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d06237344b74073a9e3ade002389744)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:01.705517 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f3de5-f730-444d-93cd-e4fd7e23223f admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:01.709455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853bc57-92c9-4a02-8919-d601c360ba3a admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:01.720695 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64f6f1-3b13-4c4f-9224-020e307dea8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:01.721651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64f6f1-3b13-4c4f-9224-020e307dea8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:01.721972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64f6f1-3b13-4c4f-9224-020e307dea8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:01.722272 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64f6f1-3b13-4c4f-9224-020e307dea8c admin admin] PATH_INFO: `/v3/projects/d72138ef91eb4a9d8d5d1862fcfa7589` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:01.723632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b64f6f1-3b13-4c4f-9224-020e307dea8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d72138ef91eb4a9d8d5d1862fcfa7589)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:01.725618 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b64f6f1-3b13-4c4f-9224-020e307dea8c admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:01.965420 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-0e7f3de5-f730-444d-93cd-e4fd7e23223f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d06237344b74073a9e3ade002389744'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:01.966818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5b64f6f1-3b13-4c4f-9224-020e307dea8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd72138ef91eb4a9d8d5d1862fcfa7589', 'request_id': 'req-5b64f6f1-3b13-4c4f-9224-020e307dea8c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:01.974241 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5b64f6f1-3b13-4c4f-9224-020e307dea8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd72138ef91eb4a9d8d5d1862fcfa7589', 'request_id': 'req-5b64f6f1-3b13-4c4f-9224-020e307dea8c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:01.980078 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5b64f6f1-3b13-4c4f-9224-020e307dea8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd72138ef91eb4a9d8d5d1862fcfa7589', 'request_id': 'req-5b64f6f1-3b13-4c4f-9224-020e307dea8c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.056750 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-0e7f3de5-f730-444d-93cd-e4fd7e23223f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d06237344b74073a9e3ade002389744'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.057492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-0e7f3de5-f730-444d-93cd-e4fd7e23223f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d06237344b74073a9e3ade002389744'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.058108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-0e7f3de5-f730-444d-93cd-e4fd7e23223f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d06237344b74073a9e3ade002389744'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.070649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-0e7f3de5-f730-444d-93cd-e4fd7e23223f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d06237344b74073a9e3ade002389744'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.083455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1334/2687] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:01 2022] DELETE /identity/v3/users/8d06237344b74073a9e3ade002389744 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:02.121435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5b64f6f1-3b13-4c4f-9224-020e307dea8c admin admin] The token cache is being invalidate because project d72138ef91eb4a9d8d5d1862fcfa7589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:02.121937 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5b64f6f1-3b13-4c4f-9224-020e307dea8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.123167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5b64f6f1-3b13-4c4f-9224-020e307dea8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.126803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1354/2688] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:01 2022] DELETE /identity/v3/projects/d72138ef91eb4a9d8d5d1862fcfa7589 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:02.133274 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f3de5-f730-444d-93cd-e4fd7e23223f admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:02.134856 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-0e7f3de5-f730-444d-93cd-e4fd7e23223f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:02.228219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f3de5-f730-444d-93cd-e4fd7e23223f admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:02.232796 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5651b1-f50d-40e3-a386-f68b79463dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:02.233704 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5651b1-f50d-40e3-a386-f68b79463dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:02.233989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5651b1-f50d-40e3-a386-f68b79463dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:02.235248 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5651b1-f50d-40e3-a386-f68b79463dda admin admin] PATH_INFO: `/v3/users/b1614bbbef9d457a9a298235fe8bca7b` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:02.236447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5651b1-f50d-40e3-a386-f68b79463dda admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1614bbbef9d457a9a298235fe8bca7b)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:02.238401 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5651b1-f50d-40e3-a386-f68b79463dda admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:02.582919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-5d5651b1-f50d-40e3-a386-f68b79463dda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1614bbbef9d457a9a298235fe8bca7b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.621591 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64f6f1-3b13-4c4f-9224-020e307dea8c admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:02.623091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5b64f6f1-3b13-4c4f-9224-020e307dea8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:02.696862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64f6f1-3b13-4c4f-9224-020e307dea8c admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:02.701732 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d7575-6730-4369-a4c8-260d376a5242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:02.702597 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d7575-6730-4369-a4c8-260d376a5242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:02.703611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d7575-6730-4369-a4c8-260d376a5242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:02.704052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d7575-6730-4369-a4c8-260d376a5242 admin admin] PATH_INFO: `/v3/projects/b35b1794fc0443af8f0733114d417d5a` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:02.705101 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d7575-6730-4369-a4c8-260d376a5242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b35b1794fc0443af8f0733114d417d5a)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:02.707022 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d7575-6730-4369-a4c8-260d376a5242 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:02.720068 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-5d5651b1-f50d-40e3-a386-f68b79463dda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1614bbbef9d457a9a298235fe8bca7b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.720734 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-5d5651b1-f50d-40e3-a386-f68b79463dda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1614bbbef9d457a9a298235fe8bca7b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.721434 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-5d5651b1-f50d-40e3-a386-f68b79463dda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1614bbbef9d457a9a298235fe8bca7b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.726888 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:02.735633 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:02.742162 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-5d5651b1-f50d-40e3-a386-f68b79463dda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1614bbbef9d457a9a298235fe8bca7b'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.757436 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1335/2689] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:02 2022] DELETE /identity/v3/users/b1614bbbef9d457a9a298235fe8bca7b => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:02.848372 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-575d7575-6730-4369-a4c8-260d376a5242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b35b1794fc0443af8f0733114d417d5a', 'request_id': 'req-575d7575-6730-4369-a4c8-260d376a5242'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.855064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-575d7575-6730-4369-a4c8-260d376a5242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b35b1794fc0443af8f0733114d417d5a', 'request_id': 'req-575d7575-6730-4369-a4c8-260d376a5242'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.861981 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-575d7575-6730-4369-a4c8-260d376a5242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b35b1794fc0443af8f0733114d417d5a', 'request_id': 'req-575d7575-6730-4369-a4c8-260d376a5242'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.970786 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-575d7575-6730-4369-a4c8-260d376a5242 admin admin] The token cache is being invalidate because project b35b1794fc0443af8f0733114d417d5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:02.971087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-575d7575-6730-4369-a4c8-260d376a5242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.971744 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-575d7575-6730-4369-a4c8-260d376a5242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:02.973352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1355/2690] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:02 2022] DELETE /identity/v3/projects/b35b1794fc0443af8f0733114d417d5a => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:02.979565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29174802-d759-4a05-96c1-a9bbe9bcbd03 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:02.979861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29174802-d759-4a05-96c1-a9bbe9bcbd03 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:02.980145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29174802-d759-4a05-96c1-a9bbe9bcbd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:02.999437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-29174802-d759-4a05-96c1-a9bbe9bcbd03 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:03.001466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-29174802-d759-4a05-96c1-a9bbe9bcbd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:03.049461 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d7575-6730-4369-a4c8-260d376a5242 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:03.049461 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-575d7575-6730-4369-a4c8-260d376a5242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:03.070608 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1336/2691] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:03.128815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d7575-6730-4369-a4c8-260d376a5242 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:03.132826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b4549-6367-41c3-8857-3940525c524e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:03.133140 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b4549-6367-41c3-8857-3940525c524e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:03.133426 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b4549-6367-41c3-8857-3940525c524e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:03.134204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f06e960-0d95-415c-9960-e5f0aded2dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:03.135211 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06e960-0d95-415c-9960-e5f0aded2dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:03.135359 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06e960-0d95-415c-9960-e5f0aded2dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:03.135647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f06e960-0d95-415c-9960-e5f0aded2dac admin admin] PATH_INFO: `/v3/projects/1e1ba892d93d4a9f8cddd5489f7334ac` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:03.136863 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06e960-0d95-415c-9960-e5f0aded2dac admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e1ba892d93d4a9f8cddd5489f7334ac)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:03.138820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f06e960-0d95-415c-9960-e5f0aded2dac admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:03.153006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b86b4549-6367-41c3-8857-3940525c524e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:03.154331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b86b4549-6367-41c3-8857-3940525c524e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:03.164514 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1337/2692] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:03:03.168609 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b4549-6367-41c3-8857-3940525c524e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:03.179071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b4549-6367-41c3-8857-3940525c524e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:03.184212 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569d78a-49e6-4d4d-9ea9-57de64deb0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:03.184921 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569d78a-49e6-4d4d-9ea9-57de64deb0eb admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:03.185237 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569d78a-49e6-4d4d-9ea9-57de64deb0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:03.185509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569d78a-49e6-4d4d-9ea9-57de64deb0eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:03.186260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569d78a-49e6-4d4d-9ea9-57de64deb0eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:03:03.186567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569d78a-49e6-4d4d-9ea9-57de64deb0eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:03.189254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569d78a-49e6-4d4d-9ea9-57de64deb0eb admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:03.204821 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1338/2693] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 11:03:03.210029 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569d78a-49e6-4d4d-9ea9-57de64deb0eb admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:03.220762 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569d78a-49e6-4d4d-9ea9-57de64deb0eb admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:03.225900 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0954fd-02c1-4b15-b0ca-78df7c3243cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:03.226587 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0954fd-02c1-4b15-b0ca-78df7c3243cf admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:03.226876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0954fd-02c1-4b15-b0ca-78df7c3243cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:03.227204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0954fd-02c1-4b15-b0ca-78df7c3243cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:03.227919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0954fd-02c1-4b15-b0ca-78df7c3243cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:03.229978 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0954fd-02c1-4b15-b0ca-78df7c3243cf admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:03.266735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5f06e960-0d95-415c-9960-e5f0aded2dac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e1ba892d93d4a9f8cddd5489f7334ac', 'request_id': 'req-5f06e960-0d95-415c-9960-e5f0aded2dac'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:03.273972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5f06e960-0d95-415c-9960-e5f0aded2dac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e1ba892d93d4a9f8cddd5489f7334ac', 'request_id': 'req-5f06e960-0d95-415c-9960-e5f0aded2dac'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:03.280107 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5f06e960-0d95-415c-9960-e5f0aded2dac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e1ba892d93d4a9f8cddd5489f7334ac', 'request_id': 'req-5f06e960-0d95-415c-9960-e5f0aded2dac'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:03.337817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1339/2694] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:03 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:03:03.342104 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0954fd-02c1-4b15-b0ca-78df7c3243cf admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:03.364348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5f06e960-0d95-415c-9960-e5f0aded2dac admin admin] The token cache is being invalidate because project 1e1ba892d93d4a9f8cddd5489f7334ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:03.364882 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5f06e960-0d95-415c-9960-e5f0aded2dac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:03.366078 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5f06e960-0d95-415c-9960-e5f0aded2dac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:03.368862 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1356/2695] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:03 2022] DELETE /identity/v3/projects/1e1ba892d93d4a9f8cddd5489f7334ac => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:03.375954 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540e2e2-3fad-420a-813a-ab95aeb83b47 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:03.381945 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540e2e2-3fad-420a-813a-ab95aeb83b47 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:03.381945 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540e2e2-3fad-420a-813a-ab95aeb83b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:03.410261 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-4540e2e2-3fad-420a-813a-ab95aeb83b47 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:03.411744 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-4540e2e2-3fad-420a-813a-ab95aeb83b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:03.424913 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0954fd-02c1-4b15-b0ca-78df7c3243cf admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:03.430116 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57316882-4797-418c-b70a-a5e5dc2de941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:03.430805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57316882-4797-418c-b70a-a5e5dc2de941 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:03.431093 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57316882-4797-418c-b70a-a5e5dc2de941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:03.431371 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57316882-4797-418c-b70a-a5e5dc2de941 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:03.432106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57316882-4797-418c-b70a-a5e5dc2de941 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:03.434082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57316882-4797-418c-b70a-a5e5dc2de941 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:03.454796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:03.489490 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1357/2696] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:03.497624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2886f7d-b496-4a3f-a081-7322f8e3e4e4 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:03.497952 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2886f7d-b496-4a3f-a081-7322f8e3e4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:03.498315 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2886f7d-b496-4a3f-a081-7322f8e3e4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:03.507019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1340/2697] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:03 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:03.512140 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57316882-4797-418c-b70a-a5e5dc2de941 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:03.514875 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-57316882-4797-418c-b70a-a5e5dc2de941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:03.517853 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-d2886f7d-b496-4a3f-a081-7322f8e3e4e4 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:03.519065 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d2886f7d-b496-4a3f-a081-7322f8e3e4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:03.535189 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1358/2698] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:03:03.537133 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57316882-4797-418c-b70a-a5e5dc2de941 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:03.539642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2886f7d-b496-4a3f-a081-7322f8e3e4e4 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:03.550754 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bbcfe-c948-4338-9ef2-727dd6ff26b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:03.550936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2886f7d-b496-4a3f-a081-7322f8e3e4e4 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:03.551448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bbcfe-c948-4338-9ef2-727dd6ff26b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:03.551726 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bbcfe-c948-4338-9ef2-727dd6ff26b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:03.552148 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bbcfe-c948-4338-9ef2-727dd6ff26b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:03.553090 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bbcfe-c948-4338-9ef2-727dd6ff26b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:03.554519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bbcfe-c948-4338-9ef2-727dd6ff26b7 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:03.556596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1802ba-3d24-45c4-8758-1898e8cb6d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:03.557520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1802ba-3d24-45c4-8758-1898e8cb6d9a admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:03.557904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1802ba-3d24-45c4-8758-1898e8cb6d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:03.558329 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1802ba-3d24-45c4-8758-1898e8cb6d9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:03.559117 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1802ba-3d24-45c4-8758-1898e8cb6d9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:03:03.559636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1802ba-3d24-45c4-8758-1898e8cb6d9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:03.561818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1802ba-3d24-45c4-8758-1898e8cb6d9a admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:03.568817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1341/2699] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:03:03.573118 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bbcfe-c948-4338-9ef2-727dd6ff26b7 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:03.579436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1359/2700] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 11:03:03.585346 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1802ba-3d24-45c4-8758-1898e8cb6d9a admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:03.589129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bbcfe-c948-4338-9ef2-727dd6ff26b7 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:03.596045 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd1da0-6659-4ae1-8077-4e37ba1f2f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:03.596688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1802ba-3d24-45c4-8758-1898e8cb6d9a admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:03.596797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd1da0-6659-4ae1-8077-4e37ba1f2f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:03.597026 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd1da0-6659-4ae1-8077-4e37ba1f2f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:03.597374 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd1da0-6659-4ae1-8077-4e37ba1f2f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:03.598116 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd1da0-6659-4ae1-8077-4e37ba1f2f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:03.599773 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd1da0-6659-4ae1-8077-4e37ba1f2f5a admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:03.601438 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaefb51-3f45-44b9-882c-47310dfcbf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:03.602096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaefb51-3f45-44b9-882c-47310dfcbf8f admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:03.603087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaefb51-3f45-44b9-882c-47310dfcbf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:03.603087 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaefb51-3f45-44b9-882c-47310dfcbf8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:03.603405 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaefb51-3f45-44b9-882c-47310dfcbf8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:03.605378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaefb51-3f45-44b9-882c-47310dfcbf8f admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:03.612086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1342/2701] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:03:03.616246 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd1da0-6659-4ae1-8077-4e37ba1f2f5a admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:03.622284 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:03.627525 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd1da0-6659-4ae1-8077-4e37ba1f2f5a admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:03.632232 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ba4ec-5547-4995-aa4c-1c93f0615065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:03.633165 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ba4ec-5547-4995-aa4c-1c93f0615065 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:03.637446 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ba4ec-5547-4995-aa4c-1c93f0615065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:03.637446 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ba4ec-5547-4995-aa4c-1c93f0615065 admin admin] PATH_INFO: `/v3/projects/9c74924f9b284e75b504f11fda7ca7f9/users/20f736f87d724b8e850ed3030e0257dd/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:03.649717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ba4ec-5547-4995-aa4c-1c93f0615065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c74924f9b284e75b504f11fda7ca7f9, user_id=20f736f87d724b8e850ed3030e0257dd, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:03.652422 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ba4ec-5547-4995-aa4c-1c93f0615065 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:03.722562 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1360/2702] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:03 2022] POST /identity/v3/projects => generated 363 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:03:03.727424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaefb51-3f45-44b9-882c-47310dfcbf8f admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:03.751947 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1343/2703] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:03 2022] PUT /identity/v3/projects/9c74924f9b284e75b504f11fda7ca7f9/users/20f736f87d724b8e850ed3030e0257dd/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:03.757839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73b385-70d4-4866-b7b8-42b88573bd4e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:03.758135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73b385-70d4-4866-b7b8-42b88573bd4e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:03.758466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73b385-70d4-4866-b7b8-42b88573bd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:03.783490 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-7c73b385-70d4-4866-b7b8-42b88573bd4e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:03.784731 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-7c73b385-70d4-4866-b7b8-42b88573bd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:03.798476 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaefb51-3f45-44b9-882c-47310dfcbf8f admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:03.864074 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1344/2704] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:03.867438 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b372684-ea1a-468b-89ec-e13f7fbe0346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:03.868614 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b372684-ea1a-468b-89ec-e13f7fbe0346 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:03.869137 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b372684-ea1a-468b-89ec-e13f7fbe0346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:03.869639 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b372684-ea1a-468b-89ec-e13f7fbe0346 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:03.870955 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b372684-ea1a-468b-89ec-e13f7fbe0346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:03.874450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73b385-70d4-4866-b7b8-42b88573bd4e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:03.875238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b372684-ea1a-468b-89ec-e13f7fbe0346 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:03.876156 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-7c73b385-70d4-4866-b7b8-42b88573bd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:03.944801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73b385-70d4-4866-b7b8-42b88573bd4e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:03.948614 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ea4f1-db5a-419e-9800-b80f8d23e9a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:03.949316 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ea4f1-db5a-419e-9800-b80f8d23e9a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:03.949632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ea4f1-db5a-419e-9800-b80f8d23e9a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:03.949887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42ea4f1-db5a-419e-9800-b80f8d23e9a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:03.961555 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42ea4f1-db5a-419e-9800-b80f8d23e9a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:03.963017 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42ea4f1-db5a-419e-9800-b80f8d23e9a1 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:03.974881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1345/2705] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 11:03:03.980699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1361/2706] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:03 2022] POST /identity/v3/users => generated 282 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:03.986730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ea4f1-db5a-419e-9800-b80f8d23e9a1 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:03.998272 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42ea4f1-db5a-419e-9800-b80f8d23e9a1 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:04.003981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241cc53-694d-4d12-b2da-dd16a5cf7696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:04.004658 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241cc53-694d-4d12-b2da-dd16a5cf7696 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:04.007625 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241cc53-694d-4d12-b2da-dd16a5cf7696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:04.007625 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241cc53-694d-4d12-b2da-dd16a5cf7696 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:04.007625 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241cc53-694d-4d12-b2da-dd16a5cf7696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:04.007625 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a241cc53-694d-4d12-b2da-dd16a5cf7696 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:04.019660 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1346/2707] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:03:04.024429 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b372684-ea1a-468b-89ec-e13f7fbe0346 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:04.036505 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b372684-ea1a-468b-89ec-e13f7fbe0346 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:04.041902 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7b23b-d5b0-4693-b931-f8db0386ab93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:04.042157 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7b23b-d5b0-4693-b931-f8db0386ab93 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:04.042473 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7b23b-d5b0-4693-b931-f8db0386ab93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:04.042754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7b23b-d5b0-4693-b931-f8db0386ab93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:04.043496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7b23b-d5b0-4693-b931-f8db0386ab93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:04.045173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7b23b-d5b0-4693-b931-f8db0386ab93 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:04.058671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1362/2708] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:04.061330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241cc53-694d-4d12-b2da-dd16a5cf7696 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:04.074210 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a241cc53-694d-4d12-b2da-dd16a5cf7696 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:04.079861 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2db83a-51fb-4bc5-9f15-88642ad96081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:04.080766 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2db83a-51fb-4bc5-9f15-88642ad96081 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:04.081082 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2db83a-51fb-4bc5-9f15-88642ad96081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:04.081421 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2db83a-51fb-4bc5-9f15-88642ad96081 admin admin] PATH_INFO: `/v3/projects/08f4a60f6b7844b98f70a1042a45c5a9/users/49cceaa42c2645209119a0c7535669c1/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:04.098158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2db83a-51fb-4bc5-9f15-88642ad96081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f4a60f6b7844b98f70a1042a45c5a9, user_id=49cceaa42c2645209119a0c7535669c1, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:04.103781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2db83a-51fb-4bc5-9f15-88642ad96081 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:04.200621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1347/2709] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:04 2022] PUT /identity/v3/projects/08f4a60f6b7844b98f70a1042a45c5a9/users/49cceaa42c2645209119a0c7535669c1/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:04.213352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cc303-4cf1-4e61-a662-fa66cca65486 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:04.213670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cc303-4cf1-4e61-a662-fa66cca65486 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:04.213951 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cc303-4cf1-4e61-a662-fa66cca65486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:04.249632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-894cc303-4cf1-4e61-a662-fa66cca65486 None None] MFA Rules not processed for user `49cceaa42c2645209119a0c7535669c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:04.251929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-894cc303-4cf1-4e61-a662-fa66cca65486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:04.272064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:04.341739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1363/2710] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:04 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:04.346145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2db83a-51fb-4bc5-9f15-88642ad96081 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:04.401379 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:04.416884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2db83a-51fb-4bc5-9f15-88642ad96081 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:04.422968 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f65544-64d7-44b4-9ed9-c5ee7fc09c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:04.423408 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f65544-64d7-44b4-9ed9-c5ee7fc09c79 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:04.424021 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f65544-64d7-44b4-9ed9-c5ee7fc09c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:04.424135 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f65544-64d7-44b4-9ed9-c5ee7fc09c79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:04.424879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f65544-64d7-44b4-9ed9-c5ee7fc09c79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:04.426853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f65544-64d7-44b4-9ed9-c5ee7fc09c79 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:04.551176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1348/2711] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:04 2022] POST /identity/v3/projects => generated 365 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:04.556778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cc303-4cf1-4e61-a662-fa66cca65486 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:04.569684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cc303-4cf1-4e61-a662-fa66cca65486 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:04.656732 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab9a10-8003-46a0-bf6a-2e3c1e4ef098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:04.657584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab9a10-8003-46a0-bf6a-2e3c1e4ef098 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:04.657873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab9a10-8003-46a0-bf6a-2e3c1e4ef098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:04.658150 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab9a10-8003-46a0-bf6a-2e3c1e4ef098 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:04.658842 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab9a10-8003-46a0-bf6a-2e3c1e4ef098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:04.660894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab9a10-8003-46a0-bf6a-2e3c1e4ef098 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:04.746631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1364/2712] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:04 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:03:04.751470 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f65544-64d7-44b4-9ed9-c5ee7fc09c79 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:04.762451 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f65544-64d7-44b4-9ed9-c5ee7fc09c79 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:04.768200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b77005-3aa5-401c-9de3-9aca194a11e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:04.768893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b77005-3aa5-401c-9de3-9aca194a11e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:04.769209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b77005-3aa5-401c-9de3-9aca194a11e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:04.769488 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b77005-3aa5-401c-9de3-9aca194a11e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:04.770236 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b77005-3aa5-401c-9de3-9aca194a11e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:04.771882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b77005-3aa5-401c-9de3-9aca194a11e3 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:04.786306 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1349/2713] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:04.789175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab9a10-8003-46a0-bf6a-2e3c1e4ef098 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:04.800734 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab9a10-8003-46a0-bf6a-2e3c1e4ef098 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:04.805570 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036005a-8c9d-43fa-9dab-15464c8bf098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:04.806512 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036005a-8c9d-43fa-9dab-15464c8bf098 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:04.806802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036005a-8c9d-43fa-9dab-15464c8bf098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:04.807076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036005a-8c9d-43fa-9dab-15464c8bf098 admin admin] PATH_INFO: `/v3/projects/995d7e569054495a8b4200daa3763b2c/users/15d51ddbfd95463ebe9ffbca3d2b7762/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:04.823874 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036005a-8c9d-43fa-9dab-15464c8bf098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=995d7e569054495a8b4200daa3763b2c, user_id=15d51ddbfd95463ebe9ffbca3d2b7762, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:04.826611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036005a-8c9d-43fa-9dab-15464c8bf098 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:04.900016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1365/2714] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:04 2022] PUT /identity/v3/projects/995d7e569054495a8b4200daa3763b2c/users/15d51ddbfd95463ebe9ffbca3d2b7762/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:04.904214 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b77005-3aa5-401c-9de3-9aca194a11e3 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:04.915147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b77005-3aa5-401c-9de3-9aca194a11e3 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:04.975044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c188b12-599d-42ed-93a0-d5c63565cc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:04.975638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c188b12-599d-42ed-93a0-d5c63565cc51 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:04.975919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c188b12-599d-42ed-93a0-d5c63565cc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:04.976400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c188b12-599d-42ed-93a0-d5c63565cc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:04.977182 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c188b12-599d-42ed-93a0-d5c63565cc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:04.978813 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c188b12-599d-42ed-93a0-d5c63565cc51 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:04.991154 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1350/2715] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:04.995722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036005a-8c9d-43fa-9dab-15464c8bf098 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:05.006831 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036005a-8c9d-43fa-9dab-15464c8bf098 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:05.011833 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb0ee9-bf96-4032-8002-dfaa39ed857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:05.012812 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb0ee9-bf96-4032-8002-dfaa39ed857c admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:05.013138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb0ee9-bf96-4032-8002-dfaa39ed857c admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:05.013423 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb0ee9-bf96-4032-8002-dfaa39ed857c admin admin] PATH_INFO: `/v3/projects/995d7e569054495a8b4200daa3763b2c/users/15d51ddbfd95463ebe9ffbca3d2b7762/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:05.015582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb0ee9-bf96-4032-8002-dfaa39ed857c admin admin] RBAC: Authorizing `identity:create_grant(project_id=995d7e569054495a8b4200daa3763b2c, user_id=15d51ddbfd95463ebe9ffbca3d2b7762, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:05.018501 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb0ee9-bf96-4032-8002-dfaa39ed857c admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:05.029527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1366/2716] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:04 2022] PUT /identity/v3/projects/995d7e569054495a8b4200daa3763b2c/users/15d51ddbfd95463ebe9ffbca3d2b7762/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:05.041284 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed0ad9-3830-4980-b07d-c0d58efc0a62 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:05.041569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed0ad9-3830-4980-b07d-c0d58efc0a62 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:05.041847 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed0ad9-3830-4980-b07d-c0d58efc0a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:05.069097 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-e6ed0ad9-3830-4980-b07d-c0d58efc0a62 None None] MFA Rules not processed for user `15d51ddbfd95463ebe9ffbca3d2b7762`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:05.070343 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e6ed0ad9-3830-4980-b07d-c0d58efc0a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:05.141543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1351/2717] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:03:05.153713 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb0ee9-bf96-4032-8002-dfaa39ed857c admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:05.155447 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-fffb0ee9-bf96-4032-8002-dfaa39ed857c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:05.214675 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb0ee9-bf96-4032-8002-dfaa39ed857c admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:05.218985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca8f6f-2988-4237-9f13-a47a98e0c7be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:05.219643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca8f6f-2988-4237-9f13-a47a98e0c7be service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:05.219929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca8f6f-2988-4237-9f13-a47a98e0c7be service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:05.220199 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca8f6f-2988-4237-9f13-a47a98e0c7be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:05.233488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca8f6f-2988-4237-9f13-a47a98e0c7be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:05.240111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca8f6f-2988-4237-9f13-a47a98e0c7be service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:05.247309 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1367/2718] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:05.331684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb46763-5eb0-4657-8382-956d54e2b451 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:05.332341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb46763-5eb0-4657-8382-956d54e2b451 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:05.332899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb46763-5eb0-4657-8382-956d54e2b451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:05.376805 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-2cb46763-5eb0-4657-8382-956d54e2b451 None None] MFA Rules not processed for user `20f736f87d724b8e850ed3030e0257dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:05.378343 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-2cb46763-5eb0-4657-8382-956d54e2b451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:05.413374 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:05.454877 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1352/2719] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:05.459148 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca8f6f-2988-4237-9f13-a47a98e0c7be service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:05.469991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:05.529673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca8f6f-2988-4237-9f13-a47a98e0c7be service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:05.536993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b730d63-791c-4f3f-888d-c1227a2d97c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:05.536993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b730d63-791c-4f3f-888d-c1227a2d97c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:05.536993 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b730d63-791c-4f3f-888d-c1227a2d97c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:05.539844 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b730d63-791c-4f3f-888d-c1227a2d97c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:05.539844 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b730d63-791c-4f3f-888d-c1227a2d97c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:05.540015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b730d63-791c-4f3f-888d-c1227a2d97c2 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:05.644611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1368/2720] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:05 2022] POST /identity/v3/projects => generated 373 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:05.645688 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb46763-5eb0-4657-8382-956d54e2b451 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:05.659942 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb46763-5eb0-4657-8382-956d54e2b451 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:05.726010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf79fb-de68-404b-a013-93276ab038c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:05.726743 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf79fb-de68-404b-a013-93276ab038c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:05.727033 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf79fb-de68-404b-a013-93276ab038c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:05.727310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf79fb-de68-404b-a013-93276ab038c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:05.728022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf79fb-de68-404b-a013-93276ab038c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:05.730221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf79fb-de68-404b-a013-93276ab038c5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:05.822412 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1353/2721] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:05 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:05.826547 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b730d63-791c-4f3f-888d-c1227a2d97c2 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:05.843530 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b730d63-791c-4f3f-888d-c1227a2d97c2 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:05.849008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8964a5-a84d-4f00-85a8-ba2653bfa813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:05.849694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8964a5-a84d-4f00-85a8-ba2653bfa813 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:05.849982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8964a5-a84d-4f00-85a8-ba2653bfa813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:05.850265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8964a5-a84d-4f00-85a8-ba2653bfa813 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:05.851018 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8964a5-a84d-4f00-85a8-ba2653bfa813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:05.852658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8964a5-a84d-4f00-85a8-ba2653bfa813 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:05.869165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1369/2722] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:05.870645 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf79fb-de68-404b-a013-93276ab038c5 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:05.882493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf79fb-de68-404b-a013-93276ab038c5 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:05.887632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffb7ab-6779-4977-8655-b238218dedca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:05.888581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffb7ab-6779-4977-8655-b238218dedca admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:05.888874 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffb7ab-6779-4977-8655-b238218dedca admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:05.889189 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffb7ab-6779-4977-8655-b238218dedca admin admin] PATH_INFO: `/v3/projects/6ca43abfadae4a17a21632e147b01392/users/19623cf21786494aa76b5034b51d444c/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:05.907619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fffb7ab-6779-4977-8655-b238218dedca admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca43abfadae4a17a21632e147b01392, user_id=19623cf21786494aa76b5034b51d444c, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:05.910306 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fffb7ab-6779-4977-8655-b238218dedca admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:06.018387 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1354/2723] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:05 2022] PUT /identity/v3/projects/6ca43abfadae4a17a21632e147b01392/users/19623cf21786494aa76b5034b51d444c/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:06.022877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8964a5-a84d-4f00-85a8-ba2653bfa813 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:06.033652 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8964a5-a84d-4f00-85a8-ba2653bfa813 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:06.095864 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521aa65-74c4-4eba-bddd-cde8b44decd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:06.096555 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521aa65-74c4-4eba-bddd-cde8b44decd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:06.096857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521aa65-74c4-4eba-bddd-cde8b44decd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:06.097147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521aa65-74c4-4eba-bddd-cde8b44decd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:06.097896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521aa65-74c4-4eba-bddd-cde8b44decd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:06.099519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521aa65-74c4-4eba-bddd-cde8b44decd6 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:06.111772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1370/2724] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:06.116044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffb7ab-6779-4977-8655-b238218dedca admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:06.127286 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffb7ab-6779-4977-8655-b238218dedca admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:06.132081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28b780-daa5-43cf-a4d6-967115e7070a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:06.134385 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28b780-daa5-43cf-a4d6-967115e7070a admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:06.134385 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28b780-daa5-43cf-a4d6-967115e7070a admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:06.134385 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28b780-daa5-43cf-a4d6-967115e7070a admin admin] PATH_INFO: `/v3/projects/6ca43abfadae4a17a21632e147b01392/users/19623cf21786494aa76b5034b51d444c/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:06.135499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28b780-daa5-43cf-a4d6-967115e7070a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca43abfadae4a17a21632e147b01392, user_id=19623cf21786494aa76b5034b51d444c, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:06.138169 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28b780-daa5-43cf-a4d6-967115e7070a admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:06.148694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1355/2725] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:06 2022] PUT /identity/v3/projects/6ca43abfadae4a17a21632e147b01392/users/19623cf21786494aa76b5034b51d444c/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:06.189222 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521aa65-74c4-4eba-bddd-cde8b44decd6 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:06.199654 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521aa65-74c4-4eba-bddd-cde8b44decd6 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:06.274034 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e7401-0b6b-4c50-a764-bd7e9c1e4474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:06.275510 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e7401-0b6b-4c50-a764-bd7e9c1e4474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:06.276041 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e7401-0b6b-4c50-a764-bd7e9c1e4474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:06.276561 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e7401-0b6b-4c50-a764-bd7e9c1e4474 admin admin] PATH_INFO: `/v3/users/49cceaa42c2645209119a0c7535669c1` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:06.278730 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e7401-0b6b-4c50-a764-bd7e9c1e4474 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49cceaa42c2645209119a0c7535669c1)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:06.282643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e7401-0b6b-4c50-a764-bd7e9c1e4474 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:06.322676 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:06.515295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-108e7401-0b6b-4c50-a764-bd7e9c1e4474 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49cceaa42c2645209119a0c7535669c1'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:06.536655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-108e7401-0b6b-4c50-a764-bd7e9c1e4474 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49cceaa42c2645209119a0c7535669c1'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:06.602581 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-108e7401-0b6b-4c50-a764-bd7e9c1e4474 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49cceaa42c2645209119a0c7535669c1'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:06.612197 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-108e7401-0b6b-4c50-a764-bd7e9c1e4474 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49cceaa42c2645209119a0c7535669c1'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:06.613181 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-108e7401-0b6b-4c50-a764-bd7e9c1e4474 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49cceaa42c2645209119a0c7535669c1'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:06.620293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1371/2726] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:06 2022] DELETE /identity/v3/users/49cceaa42c2645209119a0c7535669c1 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:06.624569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901d7a1-3be0-4e33-b09e-9dfbf337902e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:06.625014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901d7a1-3be0-4e33-b09e-9dfbf337902e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:06.625385 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2901d7a1-3be0-4e33-b09e-9dfbf337902e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:06.645838 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-2901d7a1-3be0-4e33-b09e-9dfbf337902e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:06.647057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-2901d7a1-3be0-4e33-b09e-9dfbf337902e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:06.719148 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1356/2727] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:06.739403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e7401-0b6b-4c50-a764-bd7e9c1e4474 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:06.741810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-108e7401-0b6b-4c50-a764-bd7e9c1e4474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:06.803330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e7401-0b6b-4c50-a764-bd7e9c1e4474 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:06.807642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e726f-a979-438b-89b2-9816c425b624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:06.808800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e726f-a979-438b-89b2-9816c425b624 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:06.809321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e726f-a979-438b-89b2-9816c425b624 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:06.809846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e726f-a979-438b-89b2-9816c425b624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:06.821174 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e726f-a979-438b-89b2-9816c425b624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:06.822597 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e726f-a979-438b-89b2-9816c425b624 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:06.832829 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1372/2728] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:07.446912 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901d7a1-3be0-4e33-b09e-9dfbf337902e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:07.448401 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-2901d7a1-3be0-4e33-b09e-9dfbf337902e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:07.461660 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2901d7a1-3be0-4e33-b09e-9dfbf337902e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:07.471115 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4da1d1-e39b-4266-9495-df3650d1e93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:07.472161 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4da1d1-e39b-4266-9495-df3650d1e93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:07.472358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4da1d1-e39b-4266-9495-df3650d1e93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:07.472632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4da1d1-e39b-4266-9495-df3650d1e93a admin admin] PATH_INFO: `/v3/projects/08f4a60f6b7844b98f70a1042a45c5a9` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:07.474024 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4da1d1-e39b-4266-9495-df3650d1e93a admin admin] RBAC: Authorizing `identity:delete_project(project_id=08f4a60f6b7844b98f70a1042a45c5a9)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:07.475866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4da1d1-e39b-4266-9495-df3650d1e93a admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:07.502848 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:07.593401 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba599e12-9976-42dd-8b1f-23a8660eba6a None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:07.593698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba599e12-9976-42dd-8b1f-23a8660eba6a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:07.594665 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba599e12-9976-42dd-8b1f-23a8660eba6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:07.627593 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-ba599e12-9976-42dd-8b1f-23a8660eba6a None None] MFA Rules not processed for user `19623cf21786494aa76b5034b51d444c`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:07.628883 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ba599e12-9976-42dd-8b1f-23a8660eba6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:07.635371 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6f4da1d1-e39b-4266-9495-df3650d1e93a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08f4a60f6b7844b98f70a1042a45c5a9', 'request_id': 'req-6f4da1d1-e39b-4266-9495-df3650d1e93a'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:07.643204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6f4da1d1-e39b-4266-9495-df3650d1e93a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08f4a60f6b7844b98f70a1042a45c5a9', 'request_id': 'req-6f4da1d1-e39b-4266-9495-df3650d1e93a'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:07.703478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1373/2729] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:03:07.713904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba599e12-9976-42dd-8b1f-23a8660eba6a None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:07.715313 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ba599e12-9976-42dd-8b1f-23a8660eba6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:07.728203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6f4da1d1-e39b-4266-9495-df3650d1e93a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08f4a60f6b7844b98f70a1042a45c5a9', 'request_id': 'req-6f4da1d1-e39b-4266-9495-df3650d1e93a'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:07.743884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6f4da1d1-e39b-4266-9495-df3650d1e93a admin admin] The token cache is being invalidate because project 08f4a60f6b7844b98f70a1042a45c5a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:07.744308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6f4da1d1-e39b-4266-9495-df3650d1e93a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:07.744986 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-6f4da1d1-e39b-4266-9495-df3650d1e93a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:07.746578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1357/2730] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:07 2022] DELETE /identity/v3/projects/08f4a60f6b7844b98f70a1042a45c5a9 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:07.750605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4da1d1-e39b-4266-9495-df3650d1e93a admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:07.751152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:07.751809 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-6f4da1d1-e39b-4266-9495-df3650d1e93a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:07.778019 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba599e12-9976-42dd-8b1f-23a8660eba6a None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:07.781823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afd0d5-7cdc-403b-99d1-f56af43f4a24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:07.782688 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd0d5-7cdc-403b-99d1-f56af43f4a24 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:07.782999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd0d5-7cdc-403b-99d1-f56af43f4a24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:07.783317 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd0d5-7cdc-403b-99d1-f56af43f4a24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:07.785282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c2afd0d5-7cdc-403b-99d1-f56af43f4a24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:07.818516 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4da1d1-e39b-4266-9495-df3650d1e93a admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:07.823274 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d1409-7338-4ae0-ad02-57a61d2634ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:07.824158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d1409-7338-4ae0-ad02-57a61d2634ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:07.824441 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d1409-7338-4ae0-ad02-57a61d2634ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:07.824725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d1409-7338-4ae0-ad02-57a61d2634ab admin admin] PATH_INFO: `/v3/users/15d51ddbfd95463ebe9ffbca3d2b7762` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:07.825981 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d1409-7338-4ae0-ad02-57a61d2634ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=15d51ddbfd95463ebe9ffbca3d2b7762)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:07.828557 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d1409-7338-4ae0-ad02-57a61d2634ab admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:07.849201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afd0d5-7cdc-403b-99d1-f56af43f4a24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:07.851258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afd0d5-7cdc-403b-99d1-f56af43f4a24 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:07.860896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1374/2731] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 11:03:08.025942 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-3d2d1409-7338-4ae0-ad02-57a61d2634ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15d51ddbfd95463ebe9ffbca3d2b7762'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:08.119769 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-3d2d1409-7338-4ae0-ad02-57a61d2634ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15d51ddbfd95463ebe9ffbca3d2b7762'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:08.120404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-3d2d1409-7338-4ae0-ad02-57a61d2634ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15d51ddbfd95463ebe9ffbca3d2b7762'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:08.121216 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-3d2d1409-7338-4ae0-ad02-57a61d2634ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15d51ddbfd95463ebe9ffbca3d2b7762'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:08.134813 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-3d2d1409-7338-4ae0-ad02-57a61d2634ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15d51ddbfd95463ebe9ffbca3d2b7762'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:08.147714 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1358/2732] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:07 2022] DELETE /identity/v3/users/15d51ddbfd95463ebe9ffbca3d2b7762 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:08.602759 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afd0d5-7cdc-403b-99d1-f56af43f4a24 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:08.617165 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afd0d5-7cdc-403b-99d1-f56af43f4a24 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:08.684749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687e851-5bc3-40ec-8070-eb284a07637f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:08.685689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687e851-5bc3-40ec-8070-eb284a07637f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:08.685977 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687e851-5bc3-40ec-8070-eb284a07637f admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:08.686257 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687e851-5bc3-40ec-8070-eb284a07637f admin admin] PATH_INFO: `/v3/projects/995d7e569054495a8b4200daa3763b2c` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:08.687545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687e851-5bc3-40ec-8070-eb284a07637f admin admin] RBAC: Authorizing `identity:delete_project(project_id=995d7e569054495a8b4200daa3763b2c)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:08.689609 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687e851-5bc3-40ec-8070-eb284a07637f admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:08.834910 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3687e851-5bc3-40ec-8070-eb284a07637f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '995d7e569054495a8b4200daa3763b2c', 'request_id': 'req-3687e851-5bc3-40ec-8070-eb284a07637f'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:08.841386 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3687e851-5bc3-40ec-8070-eb284a07637f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '995d7e569054495a8b4200daa3763b2c', 'request_id': 'req-3687e851-5bc3-40ec-8070-eb284a07637f'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:08.847321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3687e851-5bc3-40ec-8070-eb284a07637f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '995d7e569054495a8b4200daa3763b2c', 'request_id': 'req-3687e851-5bc3-40ec-8070-eb284a07637f'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:08.916410 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3687e851-5bc3-40ec-8070-eb284a07637f admin admin] The token cache is being invalidate because project 995d7e569054495a8b4200daa3763b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:08.917094 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3687e851-5bc3-40ec-8070-eb284a07637f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:08.917471 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3687e851-5bc3-40ec-8070-eb284a07637f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:08.919034 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1375/2733] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:08 2022] DELETE /identity/v3/projects/995d7e569054495a8b4200daa3763b2c => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:08.924095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e1471-70b1-45d5-988e-7209d21b129e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:08.927183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e1471-70b1-45d5-988e-7209d21b129e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:08.927183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e1471-70b1-45d5-988e-7209d21b129e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:08.942754 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-8f1e1471-70b1-45d5-988e-7209d21b129e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:08.943950 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-8f1e1471-70b1-45d5-988e-7209d21b129e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:08.965744 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:09.004913 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1359/2734] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:09.012881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc087592-d2fc-4b4c-848d-6f1447976077 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:09.013201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc087592-d2fc-4b4c-848d-6f1447976077 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:09.013485 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc087592-d2fc-4b4c-848d-6f1447976077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:09.032666 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-cc087592-d2fc-4b4c-848d-6f1447976077 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:09.033911 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-cc087592-d2fc-4b4c-848d-6f1447976077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:09.041857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1376/2735] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:09.046088 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e1471-70b1-45d5-988e-7209d21b129e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:09.056218 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e1471-70b1-45d5-988e-7209d21b129e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:09.060851 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244bd08-961d-424d-8841-d352feaccee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:09.061664 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244bd08-961d-424d-8841-d352feaccee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:09.061946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244bd08-961d-424d-8841-d352feaccee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:09.062229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244bd08-961d-424d-8841-d352feaccee9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:09.063522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5244bd08-961d-424d-8841-d352feaccee9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:03:09.063833 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5244bd08-961d-424d-8841-d352feaccee9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:09.065475 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5244bd08-961d-424d-8841-d352feaccee9 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:09.077167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1360/2736] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:03:09.082133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc087592-d2fc-4b4c-848d-6f1447976077 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:09.087836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:09.094657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc087592-d2fc-4b4c-848d-6f1447976077 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:09.102482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ee420-af60-47ee-a773-150e42a2a357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:09.103192 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ee420-af60-47ee-a773-150e42a2a357 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:09.103685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ee420-af60-47ee-a773-150e42a2a357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:09.104202 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ee420-af60-47ee-a773-150e42a2a357 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:09.105550 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ee420-af60-47ee-a773-150e42a2a357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:09.110082 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ee420-af60-47ee-a773-150e42a2a357 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:09.216358 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1377/2737] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:09 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:09.220768 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244bd08-961d-424d-8841-d352feaccee9 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:09.231403 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244bd08-961d-424d-8841-d352feaccee9 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:09.293234 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc3bf6-aadb-4e10-951c-2aac85e6717b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:09.293908 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc3bf6-aadb-4e10-951c-2aac85e6717b admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:09.294198 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc3bf6-aadb-4e10-951c-2aac85e6717b admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:09.294476 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc3bf6-aadb-4e10-951c-2aac85e6717b admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:09.295188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc3bf6-aadb-4e10-951c-2aac85e6717b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:09.297678 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc3bf6-aadb-4e10-951c-2aac85e6717b admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:09.367280 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1361/2738] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:09 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:09.368761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ee420-af60-47ee-a773-150e42a2a357 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:09.380522 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ee420-af60-47ee-a773-150e42a2a357 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:09.385273 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ddf2a-92d4-4e42-a51c-d16b6fac53ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:09.385935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ddf2a-92d4-4e42-a51c-d16b6fac53ac admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:09.386218 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ddf2a-92d4-4e42-a51c-d16b6fac53ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:09.386491 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ddf2a-92d4-4e42-a51c-d16b6fac53ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:09.387224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ddf2a-92d4-4e42-a51c-d16b6fac53ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:09.388834 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ddf2a-92d4-4e42-a51c-d16b6fac53ac admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:09.399512 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1378/2739] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:09.403832 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc3bf6-aadb-4e10-951c-2aac85e6717b admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:09.414244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc3bf6-aadb-4e10-951c-2aac85e6717b admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:09.418933 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba0b4c-f1aa-4767-9222-631aa3c63cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:09.419594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba0b4c-f1aa-4767-9222-631aa3c63cda admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:09.419879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba0b4c-f1aa-4767-9222-631aa3c63cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:09.420158 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba0b4c-f1aa-4767-9222-631aa3c63cda admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:09.421112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba0b4c-f1aa-4767-9222-631aa3c63cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:09.423493 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba0b4c-f1aa-4767-9222-631aa3c63cda admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:09.434144 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1362/2740] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:03:09.439551 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ddf2a-92d4-4e42-a51c-d16b6fac53ac admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:09.449742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ddf2a-92d4-4e42-a51c-d16b6fac53ac admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:09.454930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacccbf-ba39-47a6-a023-61d12282c56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:09.455892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacccbf-ba39-47a6-a023-61d12282c56e admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:09.456363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacccbf-ba39-47a6-a023-61d12282c56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:09.456745 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aacccbf-ba39-47a6-a023-61d12282c56e admin admin] PATH_INFO: `/v3/projects/a25de74b97534aab93fb6add94a1910c/users/6a0aa6b13f6a43dbaf73f6dcc59dc39c/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:09.473998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacccbf-ba39-47a6-a023-61d12282c56e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25de74b97534aab93fb6add94a1910c, user_id=6a0aa6b13f6a43dbaf73f6dcc59dc39c, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:09.476882 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aacccbf-ba39-47a6-a023-61d12282c56e admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:09.580351 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1379/2741] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:09 2022] PUT /identity/v3/projects/a25de74b97534aab93fb6add94a1910c/users/6a0aa6b13f6a43dbaf73f6dcc59dc39c/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:09.591978 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645d262-1408-4903-953b-d8993bf57ac9 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:09.592265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645d262-1408-4903-953b-d8993bf57ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:09.592543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a645d262-1408-4903-953b-d8993bf57ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:09.622547 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a645d262-1408-4903-953b-d8993bf57ac9 None None] MFA Rules not processed for user `6a0aa6b13f6a43dbaf73f6dcc59dc39c`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:09.623697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a645d262-1408-4903-953b-d8993bf57ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:09.691173 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1363/2742] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:09.695728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacccbf-ba39-47a6-a023-61d12282c56e admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:09.761249 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aacccbf-ba39-47a6-a023-61d12282c56e admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:09.767176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153b026-0338-4d3c-b39c-007c2b3026de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:09.767843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153b026-0338-4d3c-b39c-007c2b3026de admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:09.768131 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153b026-0338-4d3c-b39c-007c2b3026de admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:09.768406 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153b026-0338-4d3c-b39c-007c2b3026de admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:09.769244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153b026-0338-4d3c-b39c-007c2b3026de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:09.771438 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153b026-0338-4d3c-b39c-007c2b3026de admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:09.862374 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1380/2743] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:09 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:09.866561 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645d262-1408-4903-953b-d8993bf57ac9 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:09.878590 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a645d262-1408-4903-953b-d8993bf57ac9 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:09.899305 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:09.938432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56444e86-8abd-4b8c-a98b-cda94b6854e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:09.939110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56444e86-8abd-4b8c-a98b-cda94b6854e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:09.939398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56444e86-8abd-4b8c-a98b-cda94b6854e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:09.939675 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56444e86-8abd-4b8c-a98b-cda94b6854e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:09.940364 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56444e86-8abd-4b8c-a98b-cda94b6854e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:09.942477 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56444e86-8abd-4b8c-a98b-cda94b6854e4 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:10.057022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1364/2744] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:09 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:10.061666 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153b026-0338-4d3c-b39c-007c2b3026de admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:10.072611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153b026-0338-4d3c-b39c-007c2b3026de admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:10.077238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9218a-d765-4409-b033-7ebaf4404ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:10.078662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9218a-d765-4409-b033-7ebaf4404ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:10.079048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9218a-d765-4409-b033-7ebaf4404ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:10.079480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9218a-d765-4409-b033-7ebaf4404ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:10.080212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9218a-d765-4409-b033-7ebaf4404ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:10.081847 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9218a-d765-4409-b033-7ebaf4404ec6 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:10.092989 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1381/2745] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:03:10.097936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56444e86-8abd-4b8c-a98b-cda94b6854e4 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:10.108693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56444e86-8abd-4b8c-a98b-cda94b6854e4 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:10.114220 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783deea-6055-490a-bb97-7ef31a2fe417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:10.114892 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783deea-6055-490a-bb97-7ef31a2fe417 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:10.115176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783deea-6055-490a-bb97-7ef31a2fe417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:10.115458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783deea-6055-490a-bb97-7ef31a2fe417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:10.116196 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783deea-6055-490a-bb97-7ef31a2fe417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:10.117838 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783deea-6055-490a-bb97-7ef31a2fe417 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:10.131742 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1365/2746] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:03:10.134731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9218a-d765-4409-b033-7ebaf4404ec6 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:10.145971 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9218a-d765-4409-b033-7ebaf4404ec6 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:10.150538 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a6186-1151-4b44-987b-71caebac402f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:10.151411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a6186-1151-4b44-987b-71caebac402f admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:10.151690 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a6186-1151-4b44-987b-71caebac402f admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:10.152000 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a6186-1151-4b44-987b-71caebac402f admin admin] PATH_INFO: `/v3/projects/1a18fac42f1b4156ae254eb895af03b8/users/32ba1d4d069b40199b18c56becb82128/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:10.169582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a6186-1151-4b44-987b-71caebac402f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a18fac42f1b4156ae254eb895af03b8, user_id=32ba1d4d069b40199b18c56becb82128, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:10.172944 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a6186-1151-4b44-987b-71caebac402f admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:10.254999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1382/2747] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:10 2022] PUT /identity/v3/projects/1a18fac42f1b4156ae254eb895af03b8/users/32ba1d4d069b40199b18c56becb82128/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:10.265479 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474899bf-be9d-4207-b853-542b717fe73b None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:10.266864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474899bf-be9d-4207-b853-542b717fe73b None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:10.267210 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474899bf-be9d-4207-b853-542b717fe73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:10.294999 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-474899bf-be9d-4207-b853-542b717fe73b None None] MFA Rules not processed for user `32ba1d4d069b40199b18c56becb82128`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:10.296213 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-474899bf-be9d-4207-b853-542b717fe73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:10.373930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1366/2748] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:10.379297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a6186-1151-4b44-987b-71caebac402f admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:10.440269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:10.443785 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a6186-1151-4b44-987b-71caebac402f admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:10.449973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361d9ea-02ca-44c8-b808-9952c7e7e289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:10.450648 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361d9ea-02ca-44c8-b808-9952c7e7e289 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:10.450945 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361d9ea-02ca-44c8-b808-9952c7e7e289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:10.452664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361d9ea-02ca-44c8-b808-9952c7e7e289 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:10.452664 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361d9ea-02ca-44c8-b808-9952c7e7e289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:10.455303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361d9ea-02ca-44c8-b808-9952c7e7e289 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:10.575794 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1383/2749] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:10 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:10.580571 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474899bf-be9d-4207-b853-542b717fe73b None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:10.592208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474899bf-be9d-4207-b853-542b717fe73b None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:10.638685 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361d9ea-02ca-44c8-b808-9952c7e7e289 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:10.640590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5361d9ea-02ca-44c8-b808-9952c7e7e289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:10.686115 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a13c9-add1-4c36-9c81-68ff5998243b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:10.686876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a13c9-add1-4c36-9c81-68ff5998243b admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:10.687192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a13c9-add1-4c36-9c81-68ff5998243b admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:10.688035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a13c9-add1-4c36-9c81-68ff5998243b admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:10.689406 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a13c9-add1-4c36-9c81-68ff5998243b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:10.691396 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a13c9-add1-4c36-9c81-68ff5998243b admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:10.704947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361d9ea-02ca-44c8-b808-9952c7e7e289 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:10.708982 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fb5a1-bca5-4814-a363-f43709d1d631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:10.709672 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fb5a1-bca5-4814-a363-f43709d1d631 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:10.709960 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fb5a1-bca5-4814-a363-f43709d1d631 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:10.710242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fb5a1-bca5-4814-a363-f43709d1d631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:10.712136 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d53fb5a1-bca5-4814-a363-f43709d1d631 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:10.784299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53fb5a1-bca5-4814-a363-f43709d1d631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:10.785810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53fb5a1-bca5-4814-a363-f43709d1d631 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:10.786718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1367/2750] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:10 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:03:10.792317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a13c9-add1-4c36-9c81-68ff5998243b admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:10.797753 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1384/2751] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:10 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 11:03:10.803474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a13c9-add1-4c36-9c81-68ff5998243b admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:10.811853 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf1d36-9629-44f5-98a0-cf2513f7b470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:10.813259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf1d36-9629-44f5-98a0-cf2513f7b470 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:10.813828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf1d36-9629-44f5-98a0-cf2513f7b470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:10.814292 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf1d36-9629-44f5-98a0-cf2513f7b470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:10.815421 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf1d36-9629-44f5-98a0-cf2513f7b470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:10.817729 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf1d36-9629-44f5-98a0-cf2513f7b470 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:10.833152 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1368/2752] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:03:10.838141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fb5a1-bca5-4814-a363-f43709d1d631 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:10.848892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fb5a1-bca5-4814-a363-f43709d1d631 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:10.853689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df7341-e23c-449b-9c99-5f362a0c85d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:10.854611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df7341-e23c-449b-9c99-5f362a0c85d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:10.854895 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df7341-e23c-449b-9c99-5f362a0c85d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:10.855176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df7341-e23c-449b-9c99-5f362a0c85d3 admin admin] PATH_INFO: `/v3/projects/cb6e9741499e4cafb2f6ce12bc5cb659/users/aabfffa764b44f73a21a019a0b589df3/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:10.873915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df7341-e23c-449b-9c99-5f362a0c85d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6e9741499e4cafb2f6ce12bc5cb659, user_id=aabfffa764b44f73a21a019a0b589df3, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:10.877698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df7341-e23c-449b-9c99-5f362a0c85d3 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:11.007342 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1385/2753] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:10 2022] PUT /identity/v3/projects/cb6e9741499e4cafb2f6ce12bc5cb659/users/aabfffa764b44f73a21a019a0b589df3/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:11.014989 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf1d36-9629-44f5-98a0-cf2513f7b470 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:11.026854 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf1d36-9629-44f5-98a0-cf2513f7b470 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:11.033331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:11.087615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a489f-1bf9-443f-9cb0-67e46f0c5019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:11.088285 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a489f-1bf9-443f-9cb0-67e46f0c5019 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:11.088569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a489f-1bf9-443f-9cb0-67e46f0c5019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:11.088847 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a489f-1bf9-443f-9cb0-67e46f0c5019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:11.089650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a489f-1bf9-443f-9cb0-67e46f0c5019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:11.091241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a489f-1bf9-443f-9cb0-67e46f0c5019 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:11.101784 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1369/2754] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:03:11.106318 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df7341-e23c-449b-9c99-5f362a0c85d3 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:11.118403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df7341-e23c-449b-9c99-5f362a0c85d3 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:11.124582 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0abd1f-922e-417a-817a-09abb8daed01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:11.125584 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0abd1f-922e-417a-817a-09abb8daed01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:11.126074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0abd1f-922e-417a-817a-09abb8daed01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:11.126359 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0abd1f-922e-417a-817a-09abb8daed01 admin admin] PATH_INFO: `/v3/projects/cb6e9741499e4cafb2f6ce12bc5cb659/users/aabfffa764b44f73a21a019a0b589df3/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:11.128351 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0abd1f-922e-417a-817a-09abb8daed01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6e9741499e4cafb2f6ce12bc5cb659, user_id=aabfffa764b44f73a21a019a0b589df3, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:11.131055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0abd1f-922e-417a-817a-09abb8daed01 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:11.142258 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1386/2755] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:11 2022] PUT /identity/v3/projects/cb6e9741499e4cafb2f6ce12bc5cb659/users/aabfffa764b44f73a21a019a0b589df3/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:11.154034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438993f-1572-48f8-884e-0d5f6f44cc1c None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:11.154321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438993f-1572-48f8-884e-0d5f6f44cc1c None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:11.154597 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438993f-1572-48f8-884e-0d5f6f44cc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:11.184287 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-f438993f-1572-48f8-884e-0d5f6f44cc1c None None] MFA Rules not processed for user `aabfffa764b44f73a21a019a0b589df3`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:11.185555 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-f438993f-1572-48f8-884e-0d5f6f44cc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:11.261075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1370/2756] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:11.271991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0abd1f-922e-417a-817a-09abb8daed01 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:11.274970 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ea0abd1f-922e-417a-817a-09abb8daed01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:11.337278 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0abd1f-922e-417a-817a-09abb8daed01 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:11.340894 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0dc17-a2e3-4431-8fa7-4b2639cfc523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:11.341589 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0dc17-a2e3-4431-8fa7-4b2639cfc523 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:11.341873 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0dc17-a2e3-4431-8fa7-4b2639cfc523 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:11.342147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0dc17-a2e3-4431-8fa7-4b2639cfc523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:11.352662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0dc17-a2e3-4431-8fa7-4b2639cfc523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:11.354130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0dc17-a2e3-4431-8fa7-4b2639cfc523 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:11.358123 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:11.366328 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1387/2757] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:11.491502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438993f-1572-48f8-884e-0d5f6f44cc1c None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:11.501514 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f438993f-1572-48f8-884e-0d5f6f44cc1c None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:11.505260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd8555-8452-4746-8312-69e36a508b6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:11.505904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd8555-8452-4746-8312-69e36a508b6e service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:11.506192 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd8555-8452-4746-8312-69e36a508b6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:11.506466 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd8555-8452-4746-8312-69e36a508b6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:11.569898 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abd8555-8452-4746-8312-69e36a508b6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:11.571328 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abd8555-8452-4746-8312-69e36a508b6e service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:11.581482 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1371/2758] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:17.380771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0dc17-a2e3-4431-8fa7-4b2639cfc523 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:17.454225 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0dc17-a2e3-4431-8fa7-4b2639cfc523 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:17.460028 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded0c14-aaa0-42b3-8b8b-6e21fd1f0c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:17.460995 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded0c14-aaa0-42b3-8b8b-6e21fd1f0c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:17.461310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded0c14-aaa0-42b3-8b8b-6e21fd1f0c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:17.461588 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded0c14-aaa0-42b3-8b8b-6e21fd1f0c6c admin admin] PATH_INFO: `/v3/users/6a0aa6b13f6a43dbaf73f6dcc59dc39c` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:17.462804 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded0c14-aaa0-42b3-8b8b-6e21fd1f0c6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a0aa6b13f6a43dbaf73f6dcc59dc39c)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:17.464742 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded0c14-aaa0-42b3-8b8b-6e21fd1f0c6c admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:17.686223 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6ded0c14-aaa0-42b3-8b8b-6e21fd1f0c6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a0aa6b13f6a43dbaf73f6dcc59dc39c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:17.700756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6ded0c14-aaa0-42b3-8b8b-6e21fd1f0c6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a0aa6b13f6a43dbaf73f6dcc59dc39c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:17.796725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6ded0c14-aaa0-42b3-8b8b-6e21fd1f0c6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a0aa6b13f6a43dbaf73f6dcc59dc39c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:17.804774 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6ded0c14-aaa0-42b3-8b8b-6e21fd1f0c6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a0aa6b13f6a43dbaf73f6dcc59dc39c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:17.805498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6ded0c14-aaa0-42b3-8b8b-6e21fd1f0c6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a0aa6b13f6a43dbaf73f6dcc59dc39c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:17.811707 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1388/2759] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:17 2022] DELETE /identity/v3/users/6a0aa6b13f6a43dbaf73f6dcc59dc39c => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:17.817145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd31780-e94c-46c3-b743-d534e02b3805 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:17.817673 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd31780-e94c-46c3-b743-d534e02b3805 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:17.817673 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd31780-e94c-46c3-b743-d534e02b3805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:17.835909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-5dd31780-e94c-46c3-b743-d534e02b3805 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:17.837198 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-5dd31780-e94c-46c3-b743-d534e02b3805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:17.863704 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:17.899197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1372/2760] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:03:17.908938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded0c14-aaa0-42b3-8b8b-6e21fd1f0c6c admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:17.910380 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6ded0c14-aaa0-42b3-8b8b-6e21fd1f0c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:17.971668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded0c14-aaa0-42b3-8b8b-6e21fd1f0c6c admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:17.975448 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1444a-b401-439a-9f79-6fe9686fbfc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:17.976085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1444a-b401-439a-9f79-6fe9686fbfc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:17.976366 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1444a-b401-439a-9f79-6fe9686fbfc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:17.976643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1444a-b401-439a-9f79-6fe9686fbfc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:17.986133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1444a-b401-439a-9f79-6fe9686fbfc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:17.987828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1444a-b401-439a-9f79-6fe9686fbfc0 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:17.998105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1389/2761] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:18.523419 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd31780-e94c-46c3-b743-d534e02b3805 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:18.525277 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-5dd31780-e94c-46c3-b743-d534e02b3805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:18.542388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd31780-e94c-46c3-b743-d534e02b3805 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:18.547599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:18.548537 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:18.548862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:18.549712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9 admin admin] PATH_INFO: `/v3/projects/a25de74b97534aab93fb6add94a1910c` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:18.551475 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a25de74b97534aab93fb6add94a1910c)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:18.553486 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:18.684147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a25de74b97534aab93fb6add94a1910c', 'request_id': 'req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:18.690764 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a25de74b97534aab93fb6add94a1910c', 'request_id': 'req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:18.805408 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a25de74b97534aab93fb6add94a1910c', 'request_id': 'req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:18.817491 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9 admin admin] The token cache is being invalidate because project a25de74b97534aab93fb6add94a1910c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:18.818141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:18.818782 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:18.820381 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1373/2762] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:18 2022] DELETE /identity/v3/projects/a25de74b97534aab93fb6add94a1910c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:18.824613 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1444a-b401-439a-9f79-6fe9686fbfc0 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:18.836125 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1444a-b401-439a-9f79-6fe9686fbfc0 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:18.855380 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:18.901275 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb89198-69aa-4153-bcec-843fded3bb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:18.902153 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb89198-69aa-4153-bcec-843fded3bb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:18.902436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb89198-69aa-4153-bcec-843fded3bb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:18.902710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb89198-69aa-4153-bcec-843fded3bb9f admin admin] PATH_INFO: `/v3/users/32ba1d4d069b40199b18c56becb82128` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:18.903986 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb89198-69aa-4153-bcec-843fded3bb9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=32ba1d4d069b40199b18c56becb82128)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:18.906003 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb89198-69aa-4153-bcec-843fded3bb9f admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:19.103918 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1eb89198-69aa-4153-bcec-843fded3bb9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32ba1d4d069b40199b18c56becb82128'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:19.117071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1eb89198-69aa-4153-bcec-843fded3bb9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32ba1d4d069b40199b18c56becb82128'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:19.202591 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1eb89198-69aa-4153-bcec-843fded3bb9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32ba1d4d069b40199b18c56becb82128'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:19.211487 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1eb89198-69aa-4153-bcec-843fded3bb9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32ba1d4d069b40199b18c56becb82128'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:19.212207 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-1eb89198-69aa-4153-bcec-843fded3bb9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32ba1d4d069b40199b18c56becb82128'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:19.218532 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1390/2763] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:18 2022] DELETE /identity/v3/users/32ba1d4d069b40199b18c56becb82128 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:19.914558 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:19.916145 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:19.933396 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00374f70-1ecd-4d96-ab82-1a6fd4010fc9 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:19.993435 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49addde5-4e6d-4977-89cd-45b3f433b2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:19.994278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49addde5-4e6d-4977-89cd-45b3f433b2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:19.994520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49addde5-4e6d-4977-89cd-45b3f433b2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:19.994827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49addde5-4e6d-4977-89cd-45b3f433b2af admin admin] PATH_INFO: `/v3/projects/1a18fac42f1b4156ae254eb895af03b8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:19.996279 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49addde5-4e6d-4977-89cd-45b3f433b2af admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a18fac42f1b4156ae254eb895af03b8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:19.998172 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49addde5-4e6d-4977-89cd-45b3f433b2af admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:20.157545 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-49addde5-4e6d-4977-89cd-45b3f433b2af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a18fac42f1b4156ae254eb895af03b8', 'request_id': 'req-49addde5-4e6d-4977-89cd-45b3f433b2af'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:20.165849 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-49addde5-4e6d-4977-89cd-45b3f433b2af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a18fac42f1b4156ae254eb895af03b8', 'request_id': 'req-49addde5-4e6d-4977-89cd-45b3f433b2af'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:20.167646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:20.262924 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-49addde5-4e6d-4977-89cd-45b3f433b2af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a18fac42f1b4156ae254eb895af03b8', 'request_id': 'req-49addde5-4e6d-4977-89cd-45b3f433b2af'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:20.277225 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-49addde5-4e6d-4977-89cd-45b3f433b2af admin admin] The token cache is being invalidate because project 1a18fac42f1b4156ae254eb895af03b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:20.277572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-49addde5-4e6d-4977-89cd-45b3f433b2af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:20.278240 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-49addde5-4e6d-4977-89cd-45b3f433b2af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:20.280910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1374/2764] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:19 2022] DELETE /identity/v3/projects/1a18fac42f1b4156ae254eb895af03b8 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:20.286746 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb89198-69aa-4153-bcec-843fded3bb9f admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:20.288515 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-1eb89198-69aa-4153-bcec-843fded3bb9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:20.301585 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb89198-69aa-4153-bcec-843fded3bb9f admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:20.359684 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb72471-4d9c-4984-9a24-9c156e7d4d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:20.360829 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb72471-4d9c-4984-9a24-9c156e7d4d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:20.361150 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb72471-4d9c-4984-9a24-9c156e7d4d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:20.361435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb72471-4d9c-4984-9a24-9c156e7d4d80 admin admin] PATH_INFO: `/v3/users/aabfffa764b44f73a21a019a0b589df3` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:20.362789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb72471-4d9c-4984-9a24-9c156e7d4d80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aabfffa764b44f73a21a019a0b589df3)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:20.364791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb72471-4d9c-4984-9a24-9c156e7d4d80 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:20.573451 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:20.578655 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2bb72471-4d9c-4984-9a24-9c156e7d4d80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aabfffa764b44f73a21a019a0b589df3'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:20.590911 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2bb72471-4d9c-4984-9a24-9c156e7d4d80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aabfffa764b44f73a21a019a0b589df3'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:20.739870 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2bb72471-4d9c-4984-9a24-9c156e7d4d80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aabfffa764b44f73a21a019a0b589df3'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:20.748160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2bb72471-4d9c-4984-9a24-9c156e7d4d80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aabfffa764b44f73a21a019a0b589df3'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:20.748898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2bb72471-4d9c-4984-9a24-9c156e7d4d80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aabfffa764b44f73a21a019a0b589df3'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:20.755790 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1391/2765] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:20 2022] DELETE /identity/v3/users/aabfffa764b44f73a21a019a0b589df3 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:21.155697 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49addde5-4e6d-4977-89cd-45b3f433b2af admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:21.157222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-49addde5-4e6d-4977-89cd-45b3f433b2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:21.181031 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49addde5-4e6d-4977-89cd-45b3f433b2af admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:21.246489 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab997a-437e-4de8-a80e-2cd6e034d618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:21.247363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab997a-437e-4de8-a80e-2cd6e034d618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:21.247701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab997a-437e-4de8-a80e-2cd6e034d618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:21.248057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab997a-437e-4de8-a80e-2cd6e034d618 admin admin] PATH_INFO: `/v3/projects/cb6e9741499e4cafb2f6ce12bc5cb659` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:21.249300 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab997a-437e-4de8-a80e-2cd6e034d618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb6e9741499e4cafb2f6ce12bc5cb659)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:21.251187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab997a-437e-4de8-a80e-2cd6e034d618 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:21.386868 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-32ab997a-437e-4de8-a80e-2cd6e034d618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb6e9741499e4cafb2f6ce12bc5cb659', 'request_id': 'req-32ab997a-437e-4de8-a80e-2cd6e034d618'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:21.393371 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-32ab997a-437e-4de8-a80e-2cd6e034d618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb6e9741499e4cafb2f6ce12bc5cb659', 'request_id': 'req-32ab997a-437e-4de8-a80e-2cd6e034d618'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:21.485611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-32ab997a-437e-4de8-a80e-2cd6e034d618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb6e9741499e4cafb2f6ce12bc5cb659', 'request_id': 'req-32ab997a-437e-4de8-a80e-2cd6e034d618'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:21.497825 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-32ab997a-437e-4de8-a80e-2cd6e034d618 admin admin] The token cache is being invalidate because project cb6e9741499e4cafb2f6ce12bc5cb659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:21.498180 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-32ab997a-437e-4de8-a80e-2cd6e034d618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:21.498879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-32ab997a-437e-4de8-a80e-2cd6e034d618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:21.500432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1375/2766] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:21 2022] DELETE /identity/v3/projects/cb6e9741499e4cafb2f6ce12bc5cb659 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:21.507690 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4feb6-3a80-4706-85b0-dea166551630 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:21.508160 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4feb6-3a80-4706-85b0-dea166551630 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:21.508251 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4feb6-3a80-4706-85b0-dea166551630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:21.529269 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-f5e4feb6-3a80-4706-85b0-dea166551630 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:21.530567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f5e4feb6-3a80-4706-85b0-dea166551630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:21.606233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1392/2767] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:03:21.665642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6625b-9811-4237-830b-c22705d5ce8d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:21.665930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6625b-9811-4237-830b-c22705d5ce8d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:21.666211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6625b-9811-4237-830b-c22705d5ce8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:21.688081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a2e6625b-9811-4237-830b-c22705d5ce8d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:21.689351 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a2e6625b-9811-4237-830b-c22705d5ce8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:21.699595 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1376/2768] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:21.704147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4feb6-3a80-4706-85b0-dea166551630 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:21.715233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4feb6-3a80-4706-85b0-dea166551630 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:21.720008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea28bb-0d66-4f05-8eff-941e87da24ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:21.720689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea28bb-0d66-4f05-8eff-941e87da24ce admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:21.721007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea28bb-0d66-4f05-8eff-941e87da24ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:21.721459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea28bb-0d66-4f05-8eff-941e87da24ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:21.722175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea28bb-0d66-4f05-8eff-941e87da24ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:03:21.722354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea28bb-0d66-4f05-8eff-941e87da24ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:21.724307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea28bb-0d66-4f05-8eff-941e87da24ce admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:21.737987 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1393/2769] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:03:21.743316 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6625b-9811-4237-830b-c22705d5ce8d None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:21.753262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6625b-9811-4237-830b-c22705d5ce8d None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:21.758528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafe1df-146d-49fa-b071-7f62f3fe756c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:21.759223 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafe1df-146d-49fa-b071-7f62f3fe756c admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:21.759500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafe1df-146d-49fa-b071-7f62f3fe756c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:21.759810 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafe1df-146d-49fa-b071-7f62f3fe756c admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:21.760462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafe1df-146d-49fa-b071-7f62f3fe756c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:21.762404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafe1df-146d-49fa-b071-7f62f3fe756c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:21.847226 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1377/2770] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:21 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:21.852063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea28bb-0d66-4f05-8eff-941e87da24ce admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:21.863229 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea28bb-0d66-4f05-8eff-941e87da24ce admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:21.924435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad3492-aa83-47cb-a856-18fc0c484eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:21.925557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad3492-aa83-47cb-a856-18fc0c484eee admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:21.925816 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad3492-aa83-47cb-a856-18fc0c484eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:21.926127 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad3492-aa83-47cb-a856-18fc0c484eee admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:21.927032 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad3492-aa83-47cb-a856-18fc0c484eee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:21.929341 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad3492-aa83-47cb-a856-18fc0c484eee admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:22.062605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1394/2771] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:21 2022] POST /identity/v3/users => generated 284 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:22.066887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafe1df-146d-49fa-b071-7f62f3fe756c admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:22.077618 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafe1df-146d-49fa-b071-7f62f3fe756c admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:22.082268 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbad1dc-cb05-4295-a920-a1f594e2b6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:22.082927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbad1dc-cb05-4295-a920-a1f594e2b6a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:22.083702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbad1dc-cb05-4295-a920-a1f594e2b6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:22.084037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbad1dc-cb05-4295-a920-a1f594e2b6a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:22.084712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbad1dc-cb05-4295-a920-a1f594e2b6a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:22.086350 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbad1dc-cb05-4295-a920-a1f594e2b6a7 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:22.097778 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1378/2772] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:22.102465 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad3492-aa83-47cb-a856-18fc0c484eee admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:22.112756 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad3492-aa83-47cb-a856-18fc0c484eee admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:22.117473 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59de03-e856-4757-a754-8044b0b4e9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:22.118106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59de03-e856-4757-a754-8044b0b4e9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:22.118390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59de03-e856-4757-a754-8044b0b4e9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:22.118671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59de03-e856-4757-a754-8044b0b4e9b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:22.119432 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59de03-e856-4757-a754-8044b0b4e9b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:22.121090 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59de03-e856-4757-a754-8044b0b4e9b5 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:22.131795 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1395/2773] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:22.136308 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbad1dc-cb05-4295-a920-a1f594e2b6a7 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:22.147042 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbad1dc-cb05-4295-a920-a1f594e2b6a7 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:22.151596 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6973d20-f5ce-4b51-a469-5a641fe1119f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:22.152510 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6973d20-f5ce-4b51-a469-5a641fe1119f admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:22.152795 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6973d20-f5ce-4b51-a469-5a641fe1119f admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:22.153106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6973d20-f5ce-4b51-a469-5a641fe1119f admin admin] PATH_INFO: `/v3/projects/867e215b91bf4572b33f16af848e2759/users/913eca7ee9984fb99903af4138755a1d/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:22.168081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6973d20-f5ce-4b51-a469-5a641fe1119f admin admin] RBAC: Authorizing `identity:create_grant(project_id=867e215b91bf4572b33f16af848e2759, user_id=913eca7ee9984fb99903af4138755a1d, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:22.170836 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6973d20-f5ce-4b51-a469-5a641fe1119f admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:22.243395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1379/2774] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:22 2022] PUT /identity/v3/projects/867e215b91bf4572b33f16af848e2759/users/913eca7ee9984fb99903af4138755a1d/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:22.254697 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38236198-3639-488d-b716-557c7623d577 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:22.254992 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38236198-3639-488d-b716-557c7623d577 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:22.255758 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38236198-3639-488d-b716-557c7623d577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:22.284528 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-38236198-3639-488d-b716-557c7623d577 None None] MFA Rules not processed for user `913eca7ee9984fb99903af4138755a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:22.285775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-38236198-3639-488d-b716-557c7623d577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:22.298023 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:22.353236 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1396/2775] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:22.357819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6973d20-f5ce-4b51-a469-5a641fe1119f admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:22.368646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:22.422885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6973d20-f5ce-4b51-a469-5a641fe1119f admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:22.427884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cffac1-4f05-434b-aad6-e9cbdf6ed7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:22.428596 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cffac1-4f05-434b-aad6-e9cbdf6ed7f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:22.428885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cffac1-4f05-434b-aad6-e9cbdf6ed7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:22.429220 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cffac1-4f05-434b-aad6-e9cbdf6ed7f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:22.429970 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cffac1-4f05-434b-aad6-e9cbdf6ed7f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:22.431899 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cffac1-4f05-434b-aad6-e9cbdf6ed7f6 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:22.559334 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1380/2776] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:22 2022] POST /identity/v3/projects => generated 365 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:22.563414 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38236198-3639-488d-b716-557c7623d577 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:22.575590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38236198-3639-488d-b716-557c7623d577 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:22.632619 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc567c-95f5-4b02-936a-9eaa13f6d93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:22.633332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc567c-95f5-4b02-936a-9eaa13f6d93f admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:22.633618 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc567c-95f5-4b02-936a-9eaa13f6d93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:22.633896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc567c-95f5-4b02-936a-9eaa13f6d93f admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:22.634606 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc567c-95f5-4b02-936a-9eaa13f6d93f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:22.636567 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc567c-95f5-4b02-936a-9eaa13f6d93f admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:22.760813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1397/2777] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:22 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:22.764819 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cffac1-4f05-434b-aad6-e9cbdf6ed7f6 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:22.775810 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cffac1-4f05-434b-aad6-e9cbdf6ed7f6 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:22.780555 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf92c6a-3023-44c8-ad06-f37f3f1d9a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:22.781271 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf92c6a-3023-44c8-ad06-f37f3f1d9a33 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:22.781562 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf92c6a-3023-44c8-ad06-f37f3f1d9a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:22.781843 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf92c6a-3023-44c8-ad06-f37f3f1d9a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:22.782672 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf92c6a-3023-44c8-ad06-f37f3f1d9a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:22.784291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf92c6a-3023-44c8-ad06-f37f3f1d9a33 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:22.795690 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1381/2778] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:03:22.800295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc567c-95f5-4b02-936a-9eaa13f6d93f admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:22.811711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc567c-95f5-4b02-936a-9eaa13f6d93f admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:22.816424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429fa7e-5dbf-4cc5-9e33-6aaeb60edfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:22.818711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429fa7e-5dbf-4cc5-9e33-6aaeb60edfac admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:22.818711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429fa7e-5dbf-4cc5-9e33-6aaeb60edfac admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:22.818711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429fa7e-5dbf-4cc5-9e33-6aaeb60edfac admin admin] PATH_INFO: `/v3/projects/7c96e5076c134aef877b0e5764f9c67e/users/3924b93c35e140c4b5a610747365da35/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:22.836488 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429fa7e-5dbf-4cc5-9e33-6aaeb60edfac admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c96e5076c134aef877b0e5764f9c67e, user_id=3924b93c35e140c4b5a610747365da35, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:22.839260 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429fa7e-5dbf-4cc5-9e33-6aaeb60edfac admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:22.940378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1398/2779] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:22 2022] PUT /identity/v3/projects/7c96e5076c134aef877b0e5764f9c67e/users/3924b93c35e140c4b5a610747365da35/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:22.944395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf92c6a-3023-44c8-ad06-f37f3f1d9a33 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:22.955458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf92c6a-3023-44c8-ad06-f37f3f1d9a33 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:23.012879 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d972e80-e0f8-47e5-8139-f3221a20fc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:23.013574 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d972e80-e0f8-47e5-8139-f3221a20fc11 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:23.014009 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d972e80-e0f8-47e5-8139-f3221a20fc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:23.014234 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d972e80-e0f8-47e5-8139-f3221a20fc11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:23.014968 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d972e80-e0f8-47e5-8139-f3221a20fc11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:23.016565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d972e80-e0f8-47e5-8139-f3221a20fc11 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:23.026718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1382/2780] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:23.030871 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429fa7e-5dbf-4cc5-9e33-6aaeb60edfac admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:23.042469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429fa7e-5dbf-4cc5-9e33-6aaeb60edfac admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:23.047070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429289c5-b4a9-47d1-b52d-e2350a574657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:23.047946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429289c5-b4a9-47d1-b52d-e2350a574657 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:23.048226 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429289c5-b4a9-47d1-b52d-e2350a574657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:23.048500 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429289c5-b4a9-47d1-b52d-e2350a574657 admin admin] PATH_INFO: `/v3/projects/7c96e5076c134aef877b0e5764f9c67e/users/3924b93c35e140c4b5a610747365da35/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:23.050543 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429289c5-b4a9-47d1-b52d-e2350a574657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c96e5076c134aef877b0e5764f9c67e, user_id=3924b93c35e140c4b5a610747365da35, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:23.054464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429289c5-b4a9-47d1-b52d-e2350a574657 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:23.064819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1399/2781] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:23 2022] PUT /identity/v3/projects/7c96e5076c134aef877b0e5764f9c67e/users/3924b93c35e140c4b5a610747365da35/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:23.077103 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ba204-de53-4066-a3b3-52116c366d4e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:23.077414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ba204-de53-4066-a3b3-52116c366d4e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:23.077738 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ba204-de53-4066-a3b3-52116c366d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:23.105065 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-d34ba204-de53-4066-a3b3-52116c366d4e None None] MFA Rules not processed for user `3924b93c35e140c4b5a610747365da35`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:23.106360 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d34ba204-de53-4066-a3b3-52116c366d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:23.174385 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1383/2782] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:23 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:23.186484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429289c5-b4a9-47d1-b52d-e2350a574657 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:23.188351 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-429289c5-b4a9-47d1-b52d-e2350a574657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:23.246800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429289c5-b4a9-47d1-b52d-e2350a574657 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:23.251097 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b60629-50ef-4dc6-8b13-6a64d8c4b53e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:23.251827 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b60629-50ef-4dc6-8b13-6a64d8c4b53e service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:23.252132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b60629-50ef-4dc6-8b13-6a64d8c4b53e service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:23.252656 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b60629-50ef-4dc6-8b13-6a64d8c4b53e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:23.262816 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b60629-50ef-4dc6-8b13-6a64d8c4b53e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:23.264376 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b60629-50ef-4dc6-8b13-6a64d8c4b53e service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:23.275313 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1400/2783] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:23 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:23.480074 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ba204-de53-4066-a3b3-52116c366d4e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:23.486311 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:23.490487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ba204-de53-4066-a3b3-52116c366d4e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:23.495272 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9263993-9155-46aa-b942-aca2e752faab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:23.496012 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9263993-9155-46aa-b942-aca2e752faab service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:23.496200 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9263993-9155-46aa-b942-aca2e752faab service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:23.496473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9263993-9155-46aa-b942-aca2e752faab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:23.556919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9263993-9155-46aa-b942-aca2e752faab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:23.558414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9263993-9155-46aa-b942-aca2e752faab service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:23.567382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1384/2784] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:23 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:29.772570 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b60629-50ef-4dc6-8b13-6a64d8c4b53e service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:29.774013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-b8b60629-50ef-4dc6-8b13-6a64d8c4b53e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:29.785281 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:29.849254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b60629-50ef-4dc6-8b13-6a64d8c4b53e service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:29.854943 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc49b85-395d-4fa0-8124-6e79a126b80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:29.855772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc49b85-395d-4fa0-8124-6e79a126b80e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:29.856070 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc49b85-395d-4fa0-8124-6e79a126b80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:29.856325 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc49b85-395d-4fa0-8124-6e79a126b80e admin admin] PATH_INFO: `/v3/users/b78c09a127334196852ed9d8f67e4d07` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:29.857608 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc49b85-395d-4fa0-8124-6e79a126b80e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b78c09a127334196852ed9d8f67e4d07)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:29.859613 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc49b85-395d-4fa0-8124-6e79a126b80e admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:30.098734 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-4fc49b85-395d-4fa0-8124-6e79a126b80e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b78c09a127334196852ed9d8f67e4d07'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:30.113739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-4fc49b85-395d-4fa0-8124-6e79a126b80e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b78c09a127334196852ed9d8f67e4d07'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:30.220081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-4fc49b85-395d-4fa0-8124-6e79a126b80e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b78c09a127334196852ed9d8f67e4d07'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:30.228197 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-4fc49b85-395d-4fa0-8124-6e79a126b80e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b78c09a127334196852ed9d8f67e4d07'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:30.230418 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-4fc49b85-395d-4fa0-8124-6e79a126b80e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b78c09a127334196852ed9d8f67e4d07'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:30.235603 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1401/2785] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:29 2022] DELETE /identity/v3/users/b78c09a127334196852ed9d8f67e4d07 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:30.240725 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30e5d1-6f6d-4e82-b315-d08af409d4ed None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:30.241238 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30e5d1-6f6d-4e82-b315-d08af409d4ed None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:30.241483 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30e5d1-6f6d-4e82-b315-d08af409d4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:30.260601 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-dc30e5d1-6f6d-4e82-b315-d08af409d4ed None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:30.262241 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-dc30e5d1-6f6d-4e82-b315-d08af409d4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:30.340719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1385/2786] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:30.355153 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc49b85-395d-4fa0-8124-6e79a126b80e admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:30.357267 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-4fc49b85-395d-4fa0-8124-6e79a126b80e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:30.416794 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc49b85-395d-4fa0-8124-6e79a126b80e admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:30.421527 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90895e90-ab6f-4bef-91d5-a68a46849f95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:30.422215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90895e90-ab6f-4bef-91d5-a68a46849f95 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:30.422552 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90895e90-ab6f-4bef-91d5-a68a46849f95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:30.422876 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90895e90-ab6f-4bef-91d5-a68a46849f95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:30.432605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90895e90-ab6f-4bef-91d5-a68a46849f95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:30.434081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90895e90-ab6f-4bef-91d5-a68a46849f95 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:30.443626 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1402/2787] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:30.847410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30e5d1-6f6d-4e82-b315-d08af409d4ed None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:30.848866 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-dc30e5d1-6f6d-4e82-b315-d08af409d4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:30.861858 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30e5d1-6f6d-4e82-b315-d08af409d4ed None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:30.866550 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:30.868060 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:30.868348 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:30.868627 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd admin admin] PATH_INFO: `/v3/projects/c1b562ad06434d4590251665763b35b8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:30.869963 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1b562ad06434d4590251665763b35b8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:30.871887 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:30.975519 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1b562ad06434d4590251665763b35b8', 'request_id': 'req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:30.981997 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1b562ad06434d4590251665763b35b8', 'request_id': 'req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:31.039481 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1b562ad06434d4590251665763b35b8', 'request_id': 'req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:31.051642 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd admin admin] The token cache is being invalidate because project c1b562ad06434d4590251665763b35b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:31.051997 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:31.052951 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:31.054624 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1386/2788] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:30 2022] DELETE /identity/v3/projects/c1b562ad06434d4590251665763b35b8 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:31.059541 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90895e90-ab6f-4bef-91d5-a68a46849f95 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:31.069820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90895e90-ab6f-4bef-91d5-a68a46849f95 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:31.105061 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:31.136781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e29ad-f31b-4083-96cc-5a9d8b7ad6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:31.137765 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e29ad-f31b-4083-96cc-5a9d8b7ad6eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:31.138062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e29ad-f31b-4083-96cc-5a9d8b7ad6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:31.138322 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e29ad-f31b-4083-96cc-5a9d8b7ad6eb admin admin] PATH_INFO: `/v3/users/c6c66c940def45d78318479521768afb` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:31.139610 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e29ad-f31b-4083-96cc-5a9d8b7ad6eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c66c940def45d78318479521768afb)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:31.141637 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e29ad-f31b-4083-96cc-5a9d8b7ad6eb admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:31.328162 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a8e29ad-f31b-4083-96cc-5a9d8b7ad6eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c66c940def45d78318479521768afb'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:31.341532 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a8e29ad-f31b-4083-96cc-5a9d8b7ad6eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c66c940def45d78318479521768afb'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:31.405329 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a8e29ad-f31b-4083-96cc-5a9d8b7ad6eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c66c940def45d78318479521768afb'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:31.413132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a8e29ad-f31b-4083-96cc-5a9d8b7ad6eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c66c940def45d78318479521768afb'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:31.413785 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-5a8e29ad-f31b-4083-96cc-5a9d8b7ad6eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c66c940def45d78318479521768afb'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:31.420758 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1403/2789] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:31 2022] DELETE /identity/v3/users/c6c66c940def45d78318479521768afb => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:31.928322 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:31.928793 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:31.942966 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f63cb5-4ec6-44de-91ac-21d0d983dbbd admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:31.973416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:32.012043 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752984d9-575d-48f5-8511-c4d8ba1e6fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:32.012951 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752984d9-575d-48f5-8511-c4d8ba1e6fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:32.013270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752984d9-575d-48f5-8511-c4d8ba1e6fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:32.013546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752984d9-575d-48f5-8511-c4d8ba1e6fc2 admin admin] PATH_INFO: `/v3/projects/86bf824af415477a8cf8c9b3a799da30` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:32.014732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752984d9-575d-48f5-8511-c4d8ba1e6fc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86bf824af415477a8cf8c9b3a799da30)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:32.016665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752984d9-575d-48f5-8511-c4d8ba1e6fc2 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:32.145788 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-752984d9-575d-48f5-8511-c4d8ba1e6fc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86bf824af415477a8cf8c9b3a799da30', 'request_id': 'req-752984d9-575d-48f5-8511-c4d8ba1e6fc2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:32.154331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-752984d9-575d-48f5-8511-c4d8ba1e6fc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86bf824af415477a8cf8c9b3a799da30', 'request_id': 'req-752984d9-575d-48f5-8511-c4d8ba1e6fc2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:32.210945 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-752984d9-575d-48f5-8511-c4d8ba1e6fc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86bf824af415477a8cf8c9b3a799da30', 'request_id': 'req-752984d9-575d-48f5-8511-c4d8ba1e6fc2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:32.225208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-752984d9-575d-48f5-8511-c4d8ba1e6fc2 admin admin] The token cache is being invalidate because project 86bf824af415477a8cf8c9b3a799da30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:32.225496 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-752984d9-575d-48f5-8511-c4d8ba1e6fc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:32.226142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-752984d9-575d-48f5-8511-c4d8ba1e6fc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:32.227876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1387/2790] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:31 2022] DELETE /identity/v3/projects/86bf824af415477a8cf8c9b3a799da30 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:32.233100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b117f-0b1a-422a-927b-5c355d90b950 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:32.233617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b117f-0b1a-422a-927b-5c355d90b950 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:32.233729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b117f-0b1a-422a-927b-5c355d90b950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:32.255819 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-792b117f-0b1a-422a-927b-5c355d90b950 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:32.257085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-792b117f-0b1a-422a-927b-5c355d90b950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:32.292824 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752984d9-575d-48f5-8511-c4d8ba1e6fc2 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:32.294333 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-752984d9-575d-48f5-8511-c4d8ba1e6fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:32.360014 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1404/2791] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:32.362545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1e34e-117b-4a85-8705-9f3c739cc42c None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:32.362836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1e34e-117b-4a85-8705-9f3c739cc42c None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:32.363159 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1e34e-117b-4a85-8705-9f3c739cc42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:32.388589 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-5cf1e34e-117b-4a85-8705-9f3c739cc42c None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:32.390025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5cf1e34e-117b-4a85-8705-9f3c739cc42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:32.403299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1405/2792] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:03:32.403617 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752984d9-575d-48f5-8511-c4d8ba1e6fc2 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:32.405180 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1e34e-117b-4a85-8705-9f3c739cc42c None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:32.407340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79480da2-1c50-41b1-8bdf-c61e1654396d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:32.408357 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79480da2-1c50-41b1-8bdf-c61e1654396d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:32.408645 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79480da2-1c50-41b1-8bdf-c61e1654396d admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:32.408923 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79480da2-1c50-41b1-8bdf-c61e1654396d admin admin] PATH_INFO: `/v3/users/913eca7ee9984fb99903af4138755a1d` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:32.410250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79480da2-1c50-41b1-8bdf-c61e1654396d admin admin] RBAC: Authorizing `identity:delete_user(user_id=913eca7ee9984fb99903af4138755a1d)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:32.413340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79480da2-1c50-41b1-8bdf-c61e1654396d admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:32.416904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1e34e-117b-4a85-8705-9f3c739cc42c None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:32.422304 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c76a0-2aee-4b5f-bfcb-be5170e1f468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:32.423146 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c76a0-2aee-4b5f-bfcb-be5170e1f468 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:32.423419 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c76a0-2aee-4b5f-bfcb-be5170e1f468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:32.423772 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c76a0-2aee-4b5f-bfcb-be5170e1f468 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:32.424596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c76a0-2aee-4b5f-bfcb-be5170e1f468 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:03:32.424978 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c76a0-2aee-4b5f-bfcb-be5170e1f468 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:32.426696 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c76a0-2aee-4b5f-bfcb-be5170e1f468 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:32.445292 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1406/2793] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 11:03:32.446679 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c76a0-2aee-4b5f-bfcb-be5170e1f468 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:32.457040 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c76a0-2aee-4b5f-bfcb-be5170e1f468 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:32.462992 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877af70d-0dd4-4193-b251-366e86f7eaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:32.465629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877af70d-0dd4-4193-b251-366e86f7eaae admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:32.465629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877af70d-0dd4-4193-b251-366e86f7eaae admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:32.465629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877af70d-0dd4-4193-b251-366e86f7eaae admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:32.465629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877af70d-0dd4-4193-b251-366e86f7eaae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:32.467289 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877af70d-0dd4-4193-b251-366e86f7eaae admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:32.576651 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1407/2794] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:32 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:32.577816 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877af70d-0dd4-4193-b251-366e86f7eaae admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:32.588503 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:32.643484 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877af70d-0dd4-4193-b251-366e86f7eaae admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:32.650403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac4105-a9cc-4b1c-931a-b1f8fb7017af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:32.651091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac4105-a9cc-4b1c-931a-b1f8fb7017af admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:32.651377 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac4105-a9cc-4b1c-931a-b1f8fb7017af admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:32.651657 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac4105-a9cc-4b1c-931a-b1f8fb7017af admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:32.652361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac4105-a9cc-4b1c-931a-b1f8fb7017af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:32.654335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac4105-a9cc-4b1c-931a-b1f8fb7017af admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:32.655838 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-79480da2-1c50-41b1-8bdf-c61e1654396d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '913eca7ee9984fb99903af4138755a1d'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:32.729159 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-79480da2-1c50-41b1-8bdf-c61e1654396d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '913eca7ee9984fb99903af4138755a1d'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:32.729885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-79480da2-1c50-41b1-8bdf-c61e1654396d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '913eca7ee9984fb99903af4138755a1d'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:32.730518 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-79480da2-1c50-41b1-8bdf-c61e1654396d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '913eca7ee9984fb99903af4138755a1d'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:32.736002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:32.746329 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-79480da2-1c50-41b1-8bdf-c61e1654396d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '913eca7ee9984fb99903af4138755a1d'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:32.759812 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1388/2795] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:32 2022] DELETE /identity/v3/users/913eca7ee9984fb99903af4138755a1d => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:32.764290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e3294-e2b0-4076-9699-7ee4977e79ba None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:32.765121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e3294-e2b0-4076-9699-7ee4977e79ba None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:32.765121 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e3294-e2b0-4076-9699-7ee4977e79ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:32.765659 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1408/2796] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:32 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:32.769790 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac4105-a9cc-4b1c-931a-b1f8fb7017af admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:32.771661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-91ac4105-a9cc-4b1c-931a-b1f8fb7017af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:32.789041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-9b3e3294-e2b0-4076-9699-7ee4977e79ba None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:32.791313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9b3e3294-e2b0-4076-9699-7ee4977e79ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:32.793642 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac4105-a9cc-4b1c-931a-b1f8fb7017af admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:32.863501 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffca66-df75-478d-af0d-b9e5e506635c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:32.864474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffca66-df75-478d-af0d-b9e5e506635c admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:32.864762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffca66-df75-478d-af0d-b9e5e506635c admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:32.865078 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffca66-df75-478d-af0d-b9e5e506635c admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:32.865823 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffca66-df75-478d-af0d-b9e5e506635c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:32.868145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ffca66-df75-478d-af0d-b9e5e506635c admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:32.872086 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1389/2797] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:03:32.879703 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1409/2798] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:32.880542 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e3294-e2b0-4076-9699-7ee4977e79ba None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:32.882036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9b3e3294-e2b0-4076-9699-7ee4977e79ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:32.883462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffca66-df75-478d-af0d-b9e5e506635c admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:32.896091 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffca66-df75-478d-af0d-b9e5e506635c admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:32.901444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f0127-7d9b-4167-8592-6ed34472e806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:32.902130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f0127-7d9b-4167-8592-6ed34472e806 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:32.902437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f0127-7d9b-4167-8592-6ed34472e806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:32.902723 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f0127-7d9b-4167-8592-6ed34472e806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:32.903518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f0127-7d9b-4167-8592-6ed34472e806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:32.905242 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274f0127-7d9b-4167-8592-6ed34472e806 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:32.918364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1410/2799] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:03:32.922230 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f0127-7d9b-4167-8592-6ed34472e806 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:32.932268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f0127-7d9b-4167-8592-6ed34472e806 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:32.936940 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc6145-1c13-456b-931d-d55247cc6e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:32.937857 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc6145-1c13-456b-931d-d55247cc6e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:32.938128 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc6145-1c13-456b-931d-d55247cc6e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:32.938413 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccc6145-1c13-456b-931d-d55247cc6e94 admin admin] PATH_INFO: `/v3/projects/f0cbd9a794d6464da42ac3e163c4b1b9/users/39b3ab523c184f7f841fc0f959fa4072/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:32.951153 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e3294-e2b0-4076-9699-7ee4977e79ba None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:32.954480 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc6145-1c13-456b-931d-d55247cc6e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0cbd9a794d6464da42ac3e163c4b1b9, user_id=39b3ab523c184f7f841fc0f959fa4072, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:32.954990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775df1f-5803-4551-842f-d5698ba159da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:32.955655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775df1f-5803-4551-842f-d5698ba159da service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:32.955956 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775df1f-5803-4551-842f-d5698ba159da service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:32.956223 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775df1f-5803-4551-842f-d5698ba159da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:32.957188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccc6145-1c13-456b-931d-d55247cc6e94 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:32.967154 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775df1f-5803-4551-842f-d5698ba159da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:32.968684 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775df1f-5803-4551-842f-d5698ba159da service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:32.989374 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1390/2800] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:33.063755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1411/2801] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:32 2022] PUT /identity/v3/projects/f0cbd9a794d6464da42ac3e163c4b1b9/users/39b3ab523c184f7f841fc0f959fa4072/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:33.074246 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b602e-8ffe-4099-a879-2a0e090a8f44 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:33.074534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b602e-8ffe-4099-a879-2a0e090a8f44 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:33.074900 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b602e-8ffe-4099-a879-2a0e090a8f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:33.104319 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b74b602e-8ffe-4099-a879-2a0e090a8f44 None None] MFA Rules not processed for user `39b3ab523c184f7f841fc0f959fa4072`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:33.105575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b74b602e-8ffe-4099-a879-2a0e090a8f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:33.177764 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1391/2802] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:03:33.187287 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc6145-1c13-456b-931d-d55247cc6e94 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:33.251585 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccc6145-1c13-456b-931d-d55247cc6e94 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:33.257164 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966044c9-b369-4c0b-82f7-a11750bca364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:33.257863 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966044c9-b369-4c0b-82f7-a11750bca364 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:33.258728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966044c9-b369-4c0b-82f7-a11750bca364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:33.259008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966044c9-b369-4c0b-82f7-a11750bca364 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:33.259793 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966044c9-b369-4c0b-82f7-a11750bca364 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:33.261737 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966044c9-b369-4c0b-82f7-a11750bca364 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:33.265108 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:33.369247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b602e-8ffe-4099-a879-2a0e090a8f44 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:33.369789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1412/2803] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:33 2022] POST /identity/v3/projects => generated 375 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:33.380509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b602e-8ffe-4099-a879-2a0e090a8f44 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:33.387671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:33.448320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85371ad1-a531-4596-997d-56ec3a6c24e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:33.448928 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85371ad1-a531-4596-997d-56ec3a6c24e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:33.449262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85371ad1-a531-4596-997d-56ec3a6c24e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:33.449544 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85371ad1-a531-4596-997d-56ec3a6c24e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:33.450279 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85371ad1-a531-4596-997d-56ec3a6c24e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:33.452663 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85371ad1-a531-4596-997d-56ec3a6c24e9 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:33.583843 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966044c9-b369-4c0b-82f7-a11750bca364 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:33.584643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-966044c9-b369-4c0b-82f7-a11750bca364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:33.598322 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1392/2804] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:33 2022] POST /identity/v3/users => generated 294 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:33.600286 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85371ad1-a531-4596-997d-56ec3a6c24e9 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:33.606111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966044c9-b369-4c0b-82f7-a11750bca364 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:33.614125 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85371ad1-a531-4596-997d-56ec3a6c24e9 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:33.622841 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c414951-14b7-40c4-994d-2359445aa26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:33.623853 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c414951-14b7-40c4-994d-2359445aa26e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:33.624024 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a7dad-4ac0-48bf-94d8-6750204d5952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:33.624615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c414951-14b7-40c4-994d-2359445aa26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:33.624850 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a7dad-4ac0-48bf-94d8-6750204d5952 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:33.625047 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c414951-14b7-40c4-994d-2359445aa26e admin admin] PATH_INFO: `/v3/projects/867e215b91bf4572b33f16af848e2759` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:33.625340 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a7dad-4ac0-48bf-94d8-6750204d5952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:33.625711 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a7dad-4ac0-48bf-94d8-6750204d5952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:33.626560 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a7dad-4ac0-48bf-94d8-6750204d5952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:33.626731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c414951-14b7-40c4-994d-2359445aa26e admin admin] RBAC: Authorizing `identity:delete_project(project_id=867e215b91bf4572b33f16af848e2759)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:33.628332 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a7dad-4ac0-48bf-94d8-6750204d5952 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:33.628568 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c414951-14b7-40c4-994d-2359445aa26e admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:33.640577 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1393/2805] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:03:33.644799 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a7dad-4ac0-48bf-94d8-6750204d5952 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:33.656442 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a7dad-4ac0-48bf-94d8-6750204d5952 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:33.667956 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1153b7f-2806-44ff-a5bd-4755b353a774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:33.669414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1153b7f-2806-44ff-a5bd-4755b353a774 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:33.669804 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1153b7f-2806-44ff-a5bd-4755b353a774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:33.671294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1153b7f-2806-44ff-a5bd-4755b353a774 admin admin] PATH_INFO: `/v3/projects/6ed595bf80cb4678955ce81e9077c93b/users/c58b7013fb374cec96bb6c64791a4369/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:33.698211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1153b7f-2806-44ff-a5bd-4755b353a774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed595bf80cb4678955ce81e9077c93b, user_id=c58b7013fb374cec96bb6c64791a4369, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:33.700150 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1153b7f-2806-44ff-a5bd-4755b353a774 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:33.799114 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1394/2806] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:33 2022] PUT /identity/v3/projects/6ed595bf80cb4678955ce81e9077c93b/users/c58b7013fb374cec96bb6c64791a4369/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:33.803506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1153b7f-2806-44ff-a5bd-4755b353a774 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:33.808096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2c414951-14b7-40c4-994d-2359445aa26e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '867e215b91bf4572b33f16af848e2759', 'request_id': 'req-2c414951-14b7-40c4-994d-2359445aa26e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:33.816573 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2c414951-14b7-40c4-994d-2359445aa26e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '867e215b91bf4572b33f16af848e2759', 'request_id': 'req-2c414951-14b7-40c4-994d-2359445aa26e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:33.823611 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2c414951-14b7-40c4-994d-2359445aa26e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '867e215b91bf4572b33f16af848e2759', 'request_id': 'req-2c414951-14b7-40c4-994d-2359445aa26e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:33.874187 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1153b7f-2806-44ff-a5bd-4755b353a774 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:33.879771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2c414951-14b7-40c4-994d-2359445aa26e admin admin] The token cache is being invalidate because project 867e215b91bf4572b33f16af848e2759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:33.880110 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2c414951-14b7-40c4-994d-2359445aa26e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:33.891034 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-2c414951-14b7-40c4-994d-2359445aa26e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:33.891034 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1413/2807] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:33 2022] DELETE /identity/v3/projects/867e215b91bf4572b33f16af848e2759 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:33.891278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0d4a0-b8e5-455f-8a83-b31a2e343ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:33.891278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0d4a0-b8e5-455f-8a83-b31a2e343ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:33.891278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0d4a0-b8e5-455f-8a83-b31a2e343ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:33.891278 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0d4a0-b8e5-455f-8a83-b31a2e343ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:33.891617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c414951-14b7-40c4-994d-2359445aa26e admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:33.891702 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0d4a0-b8e5-455f-8a83-b31a2e343ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:33.891781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-2c414951-14b7-40c4-994d-2359445aa26e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:33.891862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0d4a0-b8e5-455f-8a83-b31a2e343ba8 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:33.902501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1395/2808] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:33.902501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0d4a0-b8e5-455f-8a83-b31a2e343ba8 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:33.902501 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-add0d4a0-b8e5-455f-8a83-b31a2e343ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:33.916325 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0d4a0-b8e5-455f-8a83-b31a2e343ba8 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:33.921147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1217da3-a11d-48a4-9fa9-363dd685ef21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:33.922042 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1217da3-a11d-48a4-9fa9-363dd685ef21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:33.922323 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1217da3-a11d-48a4-9fa9-363dd685ef21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:33.922605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1217da3-a11d-48a4-9fa9-363dd685ef21 admin admin] PATH_INFO: `/v3/users/3924b93c35e140c4b5a610747365da35` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:33.923808 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1217da3-a11d-48a4-9fa9-363dd685ef21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3924b93c35e140c4b5a610747365da35)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:33.925896 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1217da3-a11d-48a4-9fa9-363dd685ef21 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:33.962295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c414951-14b7-40c4-994d-2359445aa26e admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:33.967063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dabb7f-c770-4c92-a85e-7eaa90c09d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:33.967948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dabb7f-c770-4c92-a85e-7eaa90c09d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:33.968235 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dabb7f-c770-4c92-a85e-7eaa90c09d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:33.968514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dabb7f-c770-4c92-a85e-7eaa90c09d31 admin admin] PATH_INFO: `/v3/users/20f736f87d724b8e850ed3030e0257dd` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:33.969763 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dabb7f-c770-4c92-a85e-7eaa90c09d31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20f736f87d724b8e850ed3030e0257dd)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:33.971712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dabb7f-c770-4c92-a85e-7eaa90c09d31 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:34.097197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e1217da3-a11d-48a4-9fa9-363dd685ef21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3924b93c35e140c4b5a610747365da35'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.149822 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-45dabb7f-c770-4c92-a85e-7eaa90c09d31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20f736f87d724b8e850ed3030e0257dd'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.161931 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-45dabb7f-c770-4c92-a85e-7eaa90c09d31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20f736f87d724b8e850ed3030e0257dd'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.162921 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e1217da3-a11d-48a4-9fa9-363dd685ef21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3924b93c35e140c4b5a610747365da35'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.163575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e1217da3-a11d-48a4-9fa9-363dd685ef21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3924b93c35e140c4b5a610747365da35'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.164245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e1217da3-a11d-48a4-9fa9-363dd685ef21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3924b93c35e140c4b5a610747365da35'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.165735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:34.171862 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:34.178160 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-e1217da3-a11d-48a4-9fa9-363dd685ef21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3924b93c35e140c4b5a610747365da35'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.190949 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1396/2809] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:33 2022] DELETE /identity/v3/users/3924b93c35e140c4b5a610747365da35 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:34.191747 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1217da3-a11d-48a4-9fa9-363dd685ef21 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:34.193128 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e1217da3-a11d-48a4-9fa9-363dd685ef21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:34.257115 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-45dabb7f-c770-4c92-a85e-7eaa90c09d31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20f736f87d724b8e850ed3030e0257dd'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.265112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1217da3-a11d-48a4-9fa9-363dd685ef21 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:34.266350 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-45dabb7f-c770-4c92-a85e-7eaa90c09d31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20f736f87d724b8e850ed3030e0257dd'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.267299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-45dabb7f-c770-4c92-a85e-7eaa90c09d31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20f736f87d724b8e850ed3030e0257dd'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.270055 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8595a6-26a4-499c-9697-971d43612d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:34.270961 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8595a6-26a4-499c-9697-971d43612d99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:34.271254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8595a6-26a4-499c-9697-971d43612d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:34.271539 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8595a6-26a4-499c-9697-971d43612d99 admin admin] PATH_INFO: `/v3/projects/6ed595bf80cb4678955ce81e9077c93b/users/c58b7013fb374cec96bb6c64791a4369/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:34.278006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8595a6-26a4-499c-9697-971d43612d99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed595bf80cb4678955ce81e9077c93b, user_id=c58b7013fb374cec96bb6c64791a4369, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:34.278207 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8595a6-26a4-499c-9697-971d43612d99 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:34.278486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1414/2810] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:33 2022] DELETE /identity/v3/users/20f736f87d724b8e850ed3030e0257dd => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:34.293913 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1397/2811] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:34 2022] PUT /identity/v3/projects/6ed595bf80cb4678955ce81e9077c93b/users/c58b7013fb374cec96bb6c64791a4369/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:34.305960 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e95d1d-d8df-4c3d-b901-452099950b16 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:34.306521 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e95d1d-d8df-4c3d-b901-452099950b16 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:34.306904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e95d1d-d8df-4c3d-b901-452099950b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:34.335365 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-c3e95d1d-d8df-4c3d-b901-452099950b16 None None] MFA Rules not processed for user `c58b7013fb374cec96bb6c64791a4369`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:34.337378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c3e95d1d-d8df-4c3d-b901-452099950b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:34.413670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1415/2812] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:34.421889 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8595a6-26a4-499c-9697-971d43612d99 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:34.423810 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-0d8595a6-26a4-499c-9697-971d43612d99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:34.489136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8595a6-26a4-499c-9697-971d43612d99 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:34.493790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6236d-e730-45b2-b8d4-29656a71fcef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:34.494467 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6236d-e730-45b2-b8d4-29656a71fcef service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:34.494755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6236d-e730-45b2-b8d4-29656a71fcef service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:34.495034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6236d-e730-45b2-b8d4-29656a71fcef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:34.496904 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1ef6236d-e730-45b2-b8d4-29656a71fcef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:34.560293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e95d1d-d8df-4c3d-b901-452099950b16 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:34.561771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c3e95d1d-d8df-4c3d-b901-452099950b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:34.571747 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6236d-e730-45b2-b8d4-29656a71fcef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:34.573785 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6236d-e730-45b2-b8d4-29656a71fcef service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:34.584066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1398/2813] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:34.643458 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e95d1d-d8df-4c3d-b901-452099950b16 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:34.648608 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487358e6-cc29-4a14-ac25-940e33594677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:34.649605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487358e6-cc29-4a14-ac25-940e33594677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:34.650387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487358e6-cc29-4a14-ac25-940e33594677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:34.650956 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487358e6-cc29-4a14-ac25-940e33594677 admin admin] PATH_INFO: `/v3/projects/9c74924f9b284e75b504f11fda7ca7f9` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:34.653431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487358e6-cc29-4a14-ac25-940e33594677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c74924f9b284e75b504f11fda7ca7f9)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:34.657846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487358e6-cc29-4a14-ac25-940e33594677 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:34.658809 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6236d-e730-45b2-b8d4-29656a71fcef service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:34.660511 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1ef6236d-e730-45b2-b8d4-29656a71fcef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:34.677080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6236d-e730-45b2-b8d4-29656a71fcef service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:34.683222 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb3e7f-0708-4042-a7b0-6ac8047a7de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:34.684284 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb3e7f-0708-4042-a7b0-6ac8047a7de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:34.684623 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb3e7f-0708-4042-a7b0-6ac8047a7de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:34.685054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb3e7f-0708-4042-a7b0-6ac8047a7de9 admin admin] PATH_INFO: `/v3/projects/7c96e5076c134aef877b0e5764f9c67e` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:34.689084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb3e7f-0708-4042-a7b0-6ac8047a7de9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c96e5076c134aef877b0e5764f9c67e)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:34.689084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb3e7f-0708-4042-a7b0-6ac8047a7de9 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:34.778827 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-487358e6-cc29-4a14-ac25-940e33594677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c74924f9b284e75b504f11fda7ca7f9', 'request_id': 'req-487358e6-cc29-4a14-ac25-940e33594677'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.785128 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-487358e6-cc29-4a14-ac25-940e33594677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c74924f9b284e75b504f11fda7ca7f9', 'request_id': 'req-487358e6-cc29-4a14-ac25-940e33594677'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.792035 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-487358e6-cc29-4a14-ac25-940e33594677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c74924f9b284e75b504f11fda7ca7f9', 'request_id': 'req-487358e6-cc29-4a14-ac25-940e33594677'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.843668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-dbeb3e7f-0708-4042-a7b0-6ac8047a7de9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c96e5076c134aef877b0e5764f9c67e', 'request_id': 'req-dbeb3e7f-0708-4042-a7b0-6ac8047a7de9'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.850407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-dbeb3e7f-0708-4042-a7b0-6ac8047a7de9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c96e5076c134aef877b0e5764f9c67e', 'request_id': 'req-dbeb3e7f-0708-4042-a7b0-6ac8047a7de9'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.883568 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:34.911489 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-487358e6-cc29-4a14-ac25-940e33594677 admin admin] The token cache is being invalidate because project 9c74924f9b284e75b504f11fda7ca7f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:34.911783 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-487358e6-cc29-4a14-ac25-940e33594677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.912435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-487358e6-cc29-4a14-ac25-940e33594677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:34.914063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1416/2814] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:34 2022] DELETE /identity/v3/projects/9c74924f9b284e75b504f11fda7ca7f9 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:34.918680 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487358e6-cc29-4a14-ac25-940e33594677 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:34.920135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-487358e6-cc29-4a14-ac25-940e33594677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:34.925815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:34.987008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487358e6-cc29-4a14-ac25-940e33594677 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:35.005358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-dbeb3e7f-0708-4042-a7b0-6ac8047a7de9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c96e5076c134aef877b0e5764f9c67e', 'request_id': 'req-dbeb3e7f-0708-4042-a7b0-6ac8047a7de9'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:35.019920 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-dbeb3e7f-0708-4042-a7b0-6ac8047a7de9 admin admin] The token cache is being invalidate because project 7c96e5076c134aef877b0e5764f9c67e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:35.020372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-dbeb3e7f-0708-4042-a7b0-6ac8047a7de9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:35.021388 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-dbeb3e7f-0708-4042-a7b0-6ac8047a7de9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:35.023207 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1399/2815] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:34 2022] DELETE /identity/v3/projects/7c96e5076c134aef877b0e5764f9c67e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:35.029826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd345a-ced6-4e30-9ded-2f00b91d0e65 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:35.030042 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd345a-ced6-4e30-9ded-2f00b91d0e65 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:35.030326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd345a-ced6-4e30-9ded-2f00b91d0e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:35.045260 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01006336-9a97-4eb5-8c14-81d39c7950bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:35.046658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01006336-9a97-4eb5-8c14-81d39c7950bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:35.046658 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01006336-9a97-4eb5-8c14-81d39c7950bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:35.046865 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01006336-9a97-4eb5-8c14-81d39c7950bc admin admin] PATH_INFO: `/v3/users/19623cf21786494aa76b5034b51d444c` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:35.047929 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01006336-9a97-4eb5-8c14-81d39c7950bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=19623cf21786494aa76b5034b51d444c)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:35.048757 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a8bd345a-ced6-4e30-9ded-2f00b91d0e65 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:35.049932 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01006336-9a97-4eb5-8c14-81d39c7950bc admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:35.050035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a8bd345a-ced6-4e30-9ded-2f00b91d0e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:35.058621 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1400/2816] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:03:35.066323 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172e154-f9cc-4a37-a3b7-f22936035d4d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:35.066614 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172e154-f9cc-4a37-a3b7-f22936035d4d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:35.066903 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172e154-f9cc-4a37-a3b7-f22936035d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:35.084688 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-5172e154-f9cc-4a37-a3b7-f22936035d4d None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:35.085911 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-5172e154-f9cc-4a37-a3b7-f22936035d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:35.094909 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1401/2817] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:03:35.099019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172e154-f9cc-4a37-a3b7-f22936035d4d None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:35.109216 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172e154-f9cc-4a37-a3b7-f22936035d4d None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:35.114063 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fd98e-7c29-4d93-87b0-3a41c857e005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:35.114706 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fd98e-7c29-4d93-87b0-3a41c857e005 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:35.114987 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fd98e-7c29-4d93-87b0-3a41c857e005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:35.115256 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3fd98e-7c29-4d93-87b0-3a41c857e005 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:35.116061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fd98e-7c29-4d93-87b0-3a41c857e005 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:03:35.116367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fd98e-7c29-4d93-87b0-3a41c857e005 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:35.119276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3fd98e-7c29-4d93-87b0-3a41c857e005 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:35.133606 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1402/2818] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 11:03:35.136103 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fd98e-7c29-4d93-87b0-3a41c857e005 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:35.146110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3fd98e-7c29-4d93-87b0-3a41c857e005 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:35.153129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e053e7d-7587-41a4-bfb2-7e3b51aa619e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:35.153770 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e053e7d-7587-41a4-bfb2-7e3b51aa619e admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:35.156258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e053e7d-7587-41a4-bfb2-7e3b51aa619e admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:35.156258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e053e7d-7587-41a4-bfb2-7e3b51aa619e admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:35.156258 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e053e7d-7587-41a4-bfb2-7e3b51aa619e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:35.156860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e053e7d-7587-41a4-bfb2-7e3b51aa619e admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:35.253569 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1403/2819] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:35 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:03:35.257693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e053e7d-7587-41a4-bfb2-7e3b51aa619e admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:35.260514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-01006336-9a97-4eb5-8c14-81d39c7950bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19623cf21786494aa76b5034b51d444c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:35.272520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-01006336-9a97-4eb5-8c14-81d39c7950bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19623cf21786494aa76b5034b51d444c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:35.321432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e053e7d-7587-41a4-bfb2-7e3b51aa619e admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:35.326286 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdbc58-7e71-4bd3-9099-e7c6d37cd438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:35.327112 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdbc58-7e71-4bd3-9099-e7c6d37cd438 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:35.327485 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdbc58-7e71-4bd3-9099-e7c6d37cd438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:35.327873 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdbc58-7e71-4bd3-9099-e7c6d37cd438 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:35.328815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdbc58-7e71-4bd3-9099-e7c6d37cd438 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:35.331101 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdbc58-7e71-4bd3-9099-e7c6d37cd438 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:35.349497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-01006336-9a97-4eb5-8c14-81d39c7950bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19623cf21786494aa76b5034b51d444c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:35.359390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-01006336-9a97-4eb5-8c14-81d39c7950bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19623cf21786494aa76b5034b51d444c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:35.360207 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-01006336-9a97-4eb5-8c14-81d39c7950bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19623cf21786494aa76b5034b51d444c'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:35.367221 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1417/2820] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:34 2022] DELETE /identity/v3/users/19623cf21786494aa76b5034b51d444c => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:35.421317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1404/2821] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:35 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:03:35.425444 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01006336-9a97-4eb5-8c14-81d39c7950bc admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:35.427142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-01006336-9a97-4eb5-8c14-81d39c7950bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:35.499842 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01006336-9a97-4eb5-8c14-81d39c7950bc admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:35.504759 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8b99f-50b5-4a51-865f-0f2dc984f2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:35.505469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8b99f-50b5-4a51-865f-0f2dc984f2cf admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:35.505751 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8b99f-50b5-4a51-865f-0f2dc984f2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:35.506105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8b99f-50b5-4a51-865f-0f2dc984f2cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:35.506917 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8b99f-50b5-4a51-865f-0f2dc984f2cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:35.509352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8b99f-50b5-4a51-865f-0f2dc984f2cf admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:35.524879 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1418/2822] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:35.528127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdbc58-7e71-4bd3-9099-e7c6d37cd438 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:35.538900 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdbc58-7e71-4bd3-9099-e7c6d37cd438 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:35.545764 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225a3d9-e979-49a8-84e8-0acea10a0cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:35.546506 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225a3d9-e979-49a8-84e8-0acea10a0cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:35.547294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225a3d9-e979-49a8-84e8-0acea10a0cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:35.547294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225a3d9-e979-49a8-84e8-0acea10a0cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:35.549961 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225a3d9-e979-49a8-84e8-0acea10a0cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:35.549961 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225a3d9-e979-49a8-84e8-0acea10a0cf9 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:35.560334 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1405/2823] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:35.564671 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8b99f-50b5-4a51-865f-0f2dc984f2cf admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:35.578486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8b99f-50b5-4a51-865f-0f2dc984f2cf admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:35.581330 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23c302-c5d2-4275-8945-200523937484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:35.582212 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23c302-c5d2-4275-8945-200523937484 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:35.582499 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23c302-c5d2-4275-8945-200523937484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:35.582779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23c302-c5d2-4275-8945-200523937484 admin admin] PATH_INFO: `/v3/projects/d015173c58a34b0ba415c154f861fc36/users/054def6fa93240caa90cdd19c7b4e83e/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:35.600754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23c302-c5d2-4275-8945-200523937484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d015173c58a34b0ba415c154f861fc36, user_id=054def6fa93240caa90cdd19c7b4e83e, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:35.604209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23c302-c5d2-4275-8945-200523937484 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:35.633808 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225a3d9-e979-49a8-84e8-0acea10a0cf9 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:35.635549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3225a3d9-e979-49a8-84e8-0acea10a0cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:35.649801 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225a3d9-e979-49a8-84e8-0acea10a0cf9 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:35.654921 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcc27c-1686-4bbc-8768-344c5016c475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:35.655863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcc27c-1686-4bbc-8768-344c5016c475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:35.656154 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcc27c-1686-4bbc-8768-344c5016c475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:35.656433 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcc27c-1686-4bbc-8768-344c5016c475 admin admin] PATH_INFO: `/v3/projects/6ca43abfadae4a17a21632e147b01392` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:35.659320 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcc27c-1686-4bbc-8768-344c5016c475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ca43abfadae4a17a21632e147b01392)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:35.659599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcc27c-1686-4bbc-8768-344c5016c475 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:35.720120 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1419/2824] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:35 2022] PUT /identity/v3/projects/d015173c58a34b0ba415c154f861fc36/users/054def6fa93240caa90cdd19c7b4e83e/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:35.731729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fb4b8-3ae2-4ed5-92aa-a136c672ddfa None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:35.732016 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fb4b8-3ae2-4ed5-92aa-a136c672ddfa None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:35.732298 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fb4b8-3ae2-4ed5-92aa-a136c672ddfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:35.760628 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-d43fb4b8-3ae2-4ed5-92aa-a136c672ddfa None None] MFA Rules not processed for user `054def6fa93240caa90cdd19c7b4e83e`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:35.761904 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d43fb4b8-3ae2-4ed5-92aa-a136c672ddfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:35.762381 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:35.778670 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-59dcc27c-1686-4bbc-8768-344c5016c475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ca43abfadae4a17a21632e147b01392', 'request_id': 'req-59dcc27c-1686-4bbc-8768-344c5016c475'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:35.785783 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-59dcc27c-1686-4bbc-8768-344c5016c475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ca43abfadae4a17a21632e147b01392', 'request_id': 'req-59dcc27c-1686-4bbc-8768-344c5016c475'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:35.814452 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:35.832152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1420/2825] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:03:35.837122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fb4b8-3ae2-4ed5-92aa-a136c672ddfa None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:35.845817 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-59dcc27c-1686-4bbc-8768-344c5016c475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ca43abfadae4a17a21632e147b01392', 'request_id': 'req-59dcc27c-1686-4bbc-8768-344c5016c475'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:35.860820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-59dcc27c-1686-4bbc-8768-344c5016c475 admin admin] The token cache is being invalidate because project 6ca43abfadae4a17a21632e147b01392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:35.863181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-59dcc27c-1686-4bbc-8768-344c5016c475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:35.863181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-59dcc27c-1686-4bbc-8768-344c5016c475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:35.864079 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1406/2826] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:35 2022] DELETE /identity/v3/projects/6ca43abfadae4a17a21632e147b01392 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:35.926755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fb4b8-3ae2-4ed5-92aa-a136c672ddfa None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:35.999280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6c7f9-9727-4b79-9626-51b71ba86dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:35.999957 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6c7f9-9727-4b79-9626-51b71ba86dfe admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:36.000240 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6c7f9-9727-4b79-9626-51b71ba86dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:36.000517 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6c7f9-9727-4b79-9626-51b71ba86dfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:36.001400 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6c7f9-9727-4b79-9626-51b71ba86dfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:36.003470 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6c7f9-9727-4b79-9626-51b71ba86dfe admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:36.106111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1421/2827] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:35 2022] POST /identity/v3/projects => generated 381 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:03:36.110368 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcc27c-1686-4bbc-8768-344c5016c475 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:36.112512 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-59dcc27c-1686-4bbc-8768-344c5016c475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:36.126735 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcc27c-1686-4bbc-8768-344c5016c475 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:36.195645 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b25ff-3a80-4772-b99b-720161e4ce93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:36.196415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b25ff-3a80-4772-b99b-720161e4ce93 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:36.196708 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b25ff-3a80-4772-b99b-720161e4ce93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:36.197019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b25ff-3a80-4772-b99b-720161e4ce93 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:36.197229 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:36.197731 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b25ff-3a80-4772-b99b-720161e4ce93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:36.199662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b25ff-3a80-4772-b99b-720161e4ce93 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:36.291803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1407/2828] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:36 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:36.295771 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6c7f9-9727-4b79-9626-51b71ba86dfe admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:36.306366 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6c7f9-9727-4b79-9626-51b71ba86dfe admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:36.311000 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11786103-d95d-497d-92a5-47a9820fedc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:36.311786 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11786103-d95d-497d-92a5-47a9820fedc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:36.312072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11786103-d95d-497d-92a5-47a9820fedc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:36.312351 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11786103-d95d-497d-92a5-47a9820fedc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:36.313133 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11786103-d95d-497d-92a5-47a9820fedc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:36.314734 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11786103-d95d-497d-92a5-47a9820fedc9 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:36.325999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1422/2829] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:36.330218 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b25ff-3a80-4772-b99b-720161e4ce93 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:36.341633 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b25ff-3a80-4772-b99b-720161e4ce93 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:36.346464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da106a40-1cd8-4c51-8f1c-5274492d5eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:36.347347 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da106a40-1cd8-4c51-8f1c-5274492d5eec admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:36.347629 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da106a40-1cd8-4c51-8f1c-5274492d5eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:36.347906 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da106a40-1cd8-4c51-8f1c-5274492d5eec admin admin] PATH_INFO: `/v3/projects/99b5d3e7d8e24f528b0b8efe265068d2/users/fa458e5700c3443bbe3a976b0465eb28/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:36.363045 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da106a40-1cd8-4c51-8f1c-5274492d5eec admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b5d3e7d8e24f528b0b8efe265068d2, user_id=fa458e5700c3443bbe3a976b0465eb28, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:36.365646 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da106a40-1cd8-4c51-8f1c-5274492d5eec admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:36.474401 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1408/2830] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:36 2022] PUT /identity/v3/projects/99b5d3e7d8e24f528b0b8efe265068d2/users/fa458e5700c3443bbe3a976b0465eb28/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:36.478631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11786103-d95d-497d-92a5-47a9820fedc9 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:36.489668 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11786103-d95d-497d-92a5-47a9820fedc9 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:36.549595 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d1819-bbd5-45c7-9bf2-de72b25edb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:36.550310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d1819-bbd5-45c7-9bf2-de72b25edb16 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:36.550596 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d1819-bbd5-45c7-9bf2-de72b25edb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:36.550881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d1819-bbd5-45c7-9bf2-de72b25edb16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:36.551622 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d1819-bbd5-45c7-9bf2-de72b25edb16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:36.553253 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d1819-bbd5-45c7-9bf2-de72b25edb16 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:36.563589 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1423/2831] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:36.567860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da106a40-1cd8-4c51-8f1c-5274492d5eec admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:36.579461 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da106a40-1cd8-4c51-8f1c-5274492d5eec admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:36.584317 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9150618-7250-4176-a575-65348c0ad98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:36.585249 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9150618-7250-4176-a575-65348c0ad98e admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:36.585534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9150618-7250-4176-a575-65348c0ad98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:36.590210 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9150618-7250-4176-a575-65348c0ad98e admin admin] PATH_INFO: `/v3/projects/99b5d3e7d8e24f528b0b8efe265068d2/users/fa458e5700c3443bbe3a976b0465eb28/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:36.590210 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9150618-7250-4176-a575-65348c0ad98e admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b5d3e7d8e24f528b0b8efe265068d2, user_id=fa458e5700c3443bbe3a976b0465eb28, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:36.591432 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9150618-7250-4176-a575-65348c0ad98e admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:36.602334 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1409/2832] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:36 2022] PUT /identity/v3/projects/99b5d3e7d8e24f528b0b8efe265068d2/users/fa458e5700c3443bbe3a976b0465eb28/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:36.617362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bc689-211e-4895-89cd-dbad300e513e None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:36.617605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bc689-211e-4895-89cd-dbad300e513e None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:36.617886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bc689-211e-4895-89cd-dbad300e513e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:36.647033 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-445bc689-211e-4895-89cd-dbad300e513e None None] MFA Rules not processed for user `fa458e5700c3443bbe3a976b0465eb28`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:36.648303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-445bc689-211e-4895-89cd-dbad300e513e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:36.718171 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1424/2833] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:36.730015 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9150618-7250-4176-a575-65348c0ad98e admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:36.731706 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a9150618-7250-4176-a575-65348c0ad98e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:36.788867 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9150618-7250-4176-a575-65348c0ad98e admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:36.793239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4bac5-c049-4920-87d9-017321927667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:36.793893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4bac5-c049-4920-87d9-017321927667 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:36.794176 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4bac5-c049-4920-87d9-017321927667 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:36.794456 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4bac5-c049-4920-87d9-017321927667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:36.803597 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4bac5-c049-4920-87d9-017321927667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:36.805033 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4bac5-c049-4920-87d9-017321927667 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:36.815400 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1410/2834] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:36.956424 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bc689-211e-4895-89cd-dbad300e513e None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:36.966176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bc689-211e-4895-89cd-dbad300e513e None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:36.969722 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b42f3-e634-4a40-a4ff-85ebb0b7a638 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:36.970358 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b42f3-e634-4a40-a4ff-85ebb0b7a638 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:36.970661 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b42f3-e634-4a40-a4ff-85ebb0b7a638 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:36.971076 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b42f3-e634-4a40-a4ff-85ebb0b7a638 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:36.972833 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-1e9b42f3-e634-4a40-a4ff-85ebb0b7a638 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:37.005105 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:37.032678 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b42f3-e634-4a40-a4ff-85ebb0b7a638 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:37.034173 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9b42f3-e634-4a40-a4ff-85ebb0b7a638 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:37.044145 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1425/2835] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:37.499863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4bac5-c049-4920-87d9-017321927667 service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:37.513787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:37.569498 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4bac5-c049-4920-87d9-017321927667 service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:37.575221 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6115c-1783-49e0-84b0-607c836cc54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:37.576107 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6115c-1783-49e0-84b0-607c836cc54b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:37.576395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6115c-1783-49e0-84b0-607c836cc54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:37.576677 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6115c-1783-49e0-84b0-607c836cc54b admin admin] PATH_INFO: `/v3/users/054def6fa93240caa90cdd19c7b4e83e` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:37.578005 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6115c-1783-49e0-84b0-607c836cc54b admin admin] RBAC: Authorizing `identity:delete_user(user_id=054def6fa93240caa90cdd19c7b4e83e)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:37.579886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6115c-1783-49e0-84b0-607c836cc54b admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:37.762765 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-1bf6115c-1783-49e0-84b0-607c836cc54b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '054def6fa93240caa90cdd19c7b4e83e'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:37.816676 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-1bf6115c-1783-49e0-84b0-607c836cc54b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '054def6fa93240caa90cdd19c7b4e83e'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:37.817307 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-1bf6115c-1783-49e0-84b0-607c836cc54b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '054def6fa93240caa90cdd19c7b4e83e'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:37.817944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-1bf6115c-1783-49e0-84b0-607c836cc54b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '054def6fa93240caa90cdd19c7b4e83e'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:37.830967 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-1bf6115c-1783-49e0-84b0-607c836cc54b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '054def6fa93240caa90cdd19c7b4e83e'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:37.845451 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1411/2836] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:37 2022] DELETE /identity/v3/users/054def6fa93240caa90cdd19c7b4e83e => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:37.850058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117c88a-95cb-4f88-8349-028c06ef2238 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:37.850532 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117c88a-95cb-4f88-8349-028c06ef2238 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:37.850631 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117c88a-95cb-4f88-8349-028c06ef2238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:37.868704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-f117c88a-95cb-4f88-8349-028c06ef2238 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:37.869975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f117c88a-95cb-4f88-8349-028c06ef2238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:37.938250 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1426/2837] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:37.947972 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6115c-1783-49e0-84b0-607c836cc54b admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:37.950084 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-1bf6115c-1783-49e0-84b0-607c836cc54b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:38.009296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6115c-1783-49e0-84b0-607c836cc54b admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:38.013599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d28014-75d8-4fc2-be46-b62b35e9bb5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:38.014239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d28014-75d8-4fc2-be46-b62b35e9bb5f service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:38.014521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d28014-75d8-4fc2-be46-b62b35e9bb5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:38.014796 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d28014-75d8-4fc2-be46-b62b35e9bb5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:38.024472 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d28014-75d8-4fc2-be46-b62b35e9bb5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:38.025991 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d28014-75d8-4fc2-be46-b62b35e9bb5f service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:38.036360 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1412/2838] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:38.418227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117c88a-95cb-4f88-8349-028c06ef2238 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:38.420298 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f117c88a-95cb-4f88-8349-028c06ef2238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:38.433215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117c88a-95cb-4f88-8349-028c06ef2238 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:38.440139 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5070de-0cbc-44ec-b6d8-425e632971a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:38.441064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5070de-0cbc-44ec-b6d8-425e632971a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:38.441352 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5070de-0cbc-44ec-b6d8-425e632971a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:38.446795 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5070de-0cbc-44ec-b6d8-425e632971a6 admin admin] PATH_INFO: `/v3/projects/d015173c58a34b0ba415c154f861fc36` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:38.446795 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5070de-0cbc-44ec-b6d8-425e632971a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d015173c58a34b0ba415c154f861fc36)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:38.446795 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5070de-0cbc-44ec-b6d8-425e632971a6 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:38.553310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-8e5070de-0cbc-44ec-b6d8-425e632971a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd015173c58a34b0ba415c154f861fc36', 'request_id': 'req-8e5070de-0cbc-44ec-b6d8-425e632971a6'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:38.559827 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-8e5070de-0cbc-44ec-b6d8-425e632971a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd015173c58a34b0ba415c154f861fc36', 'request_id': 'req-8e5070de-0cbc-44ec-b6d8-425e632971a6'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:38.565637 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-8e5070de-0cbc-44ec-b6d8-425e632971a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd015173c58a34b0ba415c154f861fc36', 'request_id': 'req-8e5070de-0cbc-44ec-b6d8-425e632971a6'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:38.661860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-8e5070de-0cbc-44ec-b6d8-425e632971a6 admin admin] The token cache is being invalidate because project d015173c58a34b0ba415c154f861fc36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:38.662141 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-8e5070de-0cbc-44ec-b6d8-425e632971a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:38.662755 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-8e5070de-0cbc-44ec-b6d8-425e632971a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:38.664338 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1427/2839] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:38 2022] DELETE /identity/v3/projects/d015173c58a34b0ba415c154f861fc36 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:38.668934 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d28014-75d8-4fc2-be46-b62b35e9bb5f service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:38.678839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d28014-75d8-4fc2-be46-b62b35e9bb5f service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:38.708926 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:38.735884 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800979a8-49c1-46cb-a5ad-5bb2e3ab17e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:38.737054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800979a8-49c1-46cb-a5ad-5bb2e3ab17e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:38.737341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800979a8-49c1-46cb-a5ad-5bb2e3ab17e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:38.737619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800979a8-49c1-46cb-a5ad-5bb2e3ab17e6 admin admin] PATH_INFO: `/v3/users/fa458e5700c3443bbe3a976b0465eb28` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:38.738820 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800979a8-49c1-46cb-a5ad-5bb2e3ab17e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa458e5700c3443bbe3a976b0465eb28)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:38.740731 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800979a8-49c1-46cb-a5ad-5bb2e3ab17e6 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:38.957181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-800979a8-49c1-46cb-a5ad-5bb2e3ab17e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa458e5700c3443bbe3a976b0465eb28'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:39.025880 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-800979a8-49c1-46cb-a5ad-5bb2e3ab17e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa458e5700c3443bbe3a976b0465eb28'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:39.026593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-800979a8-49c1-46cb-a5ad-5bb2e3ab17e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa458e5700c3443bbe3a976b0465eb28'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:39.027234 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-800979a8-49c1-46cb-a5ad-5bb2e3ab17e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa458e5700c3443bbe3a976b0465eb28'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:39.039890 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-800979a8-49c1-46cb-a5ad-5bb2e3ab17e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa458e5700c3443bbe3a976b0465eb28'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:39.052966 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1413/2840] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:38 2022] DELETE /identity/v3/users/fa458e5700c3443bbe3a976b0465eb28 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:39.468378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5070de-0cbc-44ec-b6d8-425e632971a6 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:39.469875 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-8e5070de-0cbc-44ec-b6d8-425e632971a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:39.485881 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5070de-0cbc-44ec-b6d8-425e632971a6 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:39.545558 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:39.552448 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e46c9-8637-422e-b6b2-a331559aa039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:39.553351 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e46c9-8637-422e-b6b2-a331559aa039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:39.553634 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e46c9-8637-422e-b6b2-a331559aa039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:39.553909 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e46c9-8637-422e-b6b2-a331559aa039 admin admin] PATH_INFO: `/v3/projects/99b5d3e7d8e24f528b0b8efe265068d2` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:39.555095 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e46c9-8637-422e-b6b2-a331559aa039 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b5d3e7d8e24f528b0b8efe265068d2)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:39.557111 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e46c9-8637-422e-b6b2-a331559aa039 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:39.687202 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a80e46c9-8637-422e-b6b2-a331559aa039 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b5d3e7d8e24f528b0b8efe265068d2', 'request_id': 'req-a80e46c9-8637-422e-b6b2-a331559aa039'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:39.693357 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a80e46c9-8637-422e-b6b2-a331559aa039 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b5d3e7d8e24f528b0b8efe265068d2', 'request_id': 'req-a80e46c9-8637-422e-b6b2-a331559aa039'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:39.699727 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a80e46c9-8637-422e-b6b2-a331559aa039 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b5d3e7d8e24f528b0b8efe265068d2', 'request_id': 'req-a80e46c9-8637-422e-b6b2-a331559aa039'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:39.770820 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a80e46c9-8637-422e-b6b2-a331559aa039 admin admin] The token cache is being invalidate because project 99b5d3e7d8e24f528b0b8efe265068d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:39.771113 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a80e46c9-8637-422e-b6b2-a331559aa039 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:39.771726 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-a80e46c9-8637-422e-b6b2-a331559aa039 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:39.773806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1428/2841] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:39 2022] DELETE /identity/v3/projects/99b5d3e7d8e24f528b0b8efe265068d2 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:39.780761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95201a-8c8c-44a5-b3b7-108812bdacd7 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:39.781256 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95201a-8c8c-44a5-b3b7-108812bdacd7 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:39.781341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95201a-8c8c-44a5-b3b7-108812bdacd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:39.799265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-0b95201a-8c8c-44a5-b3b7-108812bdacd7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:39.800500 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-0b95201a-8c8c-44a5-b3b7-108812bdacd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:39.861811 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1414/2842] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:03:39.869784 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c829642-42e1-4928-82bb-5f41aa027314 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:39.870075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c829642-42e1-4928-82bb-5f41aa027314 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:39.871453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c829642-42e1-4928-82bb-5f41aa027314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:39.891008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-8c829642-42e1-4928-82bb-5f41aa027314 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:39.892209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-8c829642-42e1-4928-82bb-5f41aa027314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:39.900072 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1429/2843] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:39.905183 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95201a-8c8c-44a5-b3b7-108812bdacd7 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:39.914946 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95201a-8c8c-44a5-b3b7-108812bdacd7 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:39.919656 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b488a66-5544-4c97-a094-629cf2bb1713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:39.920333 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b488a66-5544-4c97-a094-629cf2bb1713 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:39.920615 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b488a66-5544-4c97-a094-629cf2bb1713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:39.920885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b488a66-5544-4c97-a094-629cf2bb1713 admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:39.921640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b488a66-5544-4c97-a094-629cf2bb1713 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:03:39.921944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b488a66-5544-4c97-a094-629cf2bb1713 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:39.923524 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b488a66-5544-4c97-a094-629cf2bb1713 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:39.936031 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1415/2844] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 11:03:39.940457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c829642-42e1-4928-82bb-5f41aa027314 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:39.950818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c829642-42e1-4928-82bb-5f41aa027314 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:39.957042 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68da96-733a-4683-95f8-f8bf21f34de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:39.957709 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68da96-733a-4683-95f8-f8bf21f34de3 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:39.957998 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68da96-733a-4683-95f8-f8bf21f34de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:39.958271 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68da96-733a-4683-95f8-f8bf21f34de3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:39.958962 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68da96-733a-4683-95f8-f8bf21f34de3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:39.960901 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68da96-733a-4683-95f8-f8bf21f34de3 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:40.065481 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1430/2845] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:39 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:40.069598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b488a66-5544-4c97-a094-629cf2bb1713 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:40.079921 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b488a66-5544-4c97-a094-629cf2bb1713 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:40.118580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:40.137504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9527824-d301-4f69-8a63-beabe4e9b037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:40.138179 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9527824-d301-4f69-8a63-beabe4e9b037 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:40.138469 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9527824-d301-4f69-8a63-beabe4e9b037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:40.138752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9527824-d301-4f69-8a63-beabe4e9b037 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:40.139449 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9527824-d301-4f69-8a63-beabe4e9b037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:40.141443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9527824-d301-4f69-8a63-beabe4e9b037 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:40.225860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1416/2846] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:40 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:40.230765 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68da96-733a-4683-95f8-f8bf21f34de3 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:40.241832 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68da96-733a-4683-95f8-f8bf21f34de3 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:40.247385 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c231f55-fdf5-4a72-93bf-536d1c5acc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:40.248048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c231f55-fdf5-4a72-93bf-536d1c5acc87 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:40.248335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c231f55-fdf5-4a72-93bf-536d1c5acc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:40.248620 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c231f55-fdf5-4a72-93bf-536d1c5acc87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:40.249439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c231f55-fdf5-4a72-93bf-536d1c5acc87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:40.251045 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c231f55-fdf5-4a72-93bf-536d1c5acc87 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:40.262057 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1431/2847] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:40.266103 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9527824-d301-4f69-8a63-beabe4e9b037 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:40.276081 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9527824-d301-4f69-8a63-beabe4e9b037 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:40.280781 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0cf77-865f-41cf-9fe7-b4ff5fb46f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:40.281470 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0cf77-865f-41cf-9fe7-b4ff5fb46f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:40.281752 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0cf77-865f-41cf-9fe7-b4ff5fb46f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:40.282027 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0cf77-865f-41cf-9fe7-b4ff5fb46f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:40.282828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0cf77-865f-41cf-9fe7-b4ff5fb46f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:40.284537 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0cf77-865f-41cf-9fe7-b4ff5fb46f4d admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:40.295106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1417/2848] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:03:40.299371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c231f55-fdf5-4a72-93bf-536d1c5acc87 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:40.310096 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c231f55-fdf5-4a72-93bf-536d1c5acc87 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:40.314796 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf30c18-482b-472e-ab7a-b5dbb4f374d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:40.315721 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf30c18-482b-472e-ab7a-b5dbb4f374d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:40.315948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf30c18-482b-472e-ab7a-b5dbb4f374d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:40.316231 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf30c18-482b-472e-ab7a-b5dbb4f374d5 admin admin] PATH_INFO: `/v3/projects/31cb7b8ee8ac4e23961595d2d0336cff/users/0aba9e39520a42d29989943938c3e0e0/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:40.331916 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf30c18-482b-472e-ab7a-b5dbb4f374d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31cb7b8ee8ac4e23961595d2d0336cff, user_id=0aba9e39520a42d29989943938c3e0e0, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:40.334614 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf30c18-482b-472e-ab7a-b5dbb4f374d5 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:40.393775 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1432/2849] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:40 2022] PUT /identity/v3/projects/31cb7b8ee8ac4e23961595d2d0336cff/users/0aba9e39520a42d29989943938c3e0e0/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:40.406184 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e9881-1a68-4b5c-8294-20944e1a271d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:40.406521 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e9881-1a68-4b5c-8294-20944e1a271d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:40.406802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e9881-1a68-4b5c-8294-20944e1a271d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:40.434492 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-619e9881-1a68-4b5c-8294-20944e1a271d None None] MFA Rules not processed for user `0aba9e39520a42d29989943938c3e0e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:40.435693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-619e9881-1a68-4b5c-8294-20944e1a271d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:40.499896 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1418/2850] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:40.504545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf30c18-482b-472e-ab7a-b5dbb4f374d5 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:40.570836 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf30c18-482b-472e-ab7a-b5dbb4f374d5 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:40.576209 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814cbff-14ae-43f4-8c53-4e9f0ba457df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:40.576864 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814cbff-14ae-43f4-8c53-4e9f0ba457df admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:40.577174 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814cbff-14ae-43f4-8c53-4e9f0ba457df admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:40.577449 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814cbff-14ae-43f4-8c53-4e9f0ba457df admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:40.578210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b814cbff-14ae-43f4-8c53-4e9f0ba457df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:40.580175 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b814cbff-14ae-43f4-8c53-4e9f0ba457df admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:40.679495 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1433/2851] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:40 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:40.683556 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e9881-1a68-4b5c-8294-20944e1a271d None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:40.696546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e9881-1a68-4b5c-8294-20944e1a271d None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:40.759354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb2c62-3679-4f32-8b45-6cfa831d8f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:40.760024 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb2c62-3679-4f32-8b45-6cfa831d8f53 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:40.760307 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb2c62-3679-4f32-8b45-6cfa831d8f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:40.760580 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb2c62-3679-4f32-8b45-6cfa831d8f53 admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:40.761960 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb2c62-3679-4f32-8b45-6cfa831d8f53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:40.764061 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb2c62-3679-4f32-8b45-6cfa831d8f53 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:40.827319 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1419/2852] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:40 2022] POST /identity/v3/users => generated 294 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:40.831497 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814cbff-14ae-43f4-8c53-4e9f0ba457df admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:40.843818 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814cbff-14ae-43f4-8c53-4e9f0ba457df admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:40.848710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa1135-9cb3-4c09-92ae-17905aa039c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:40.849448 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa1135-9cb3-4c09-92ae-17905aa039c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:40.849739 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa1135-9cb3-4c09-92ae-17905aa039c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:40.851025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa1135-9cb3-4c09-92ae-17905aa039c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:40.851025 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa1135-9cb3-4c09-92ae-17905aa039c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:40.852373 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa1135-9cb3-4c09-92ae-17905aa039c3 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:40.854009 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:40.864835 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1434/2853] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:40.869643 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb2c62-3679-4f32-8b45-6cfa831d8f53 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:40.880008 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb2c62-3679-4f32-8b45-6cfa831d8f53 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:40.884718 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b410c78-5d16-4eca-b248-26ffa998c5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:40.885637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b410c78-5d16-4eca-b248-26ffa998c5b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:40.886437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b410c78-5d16-4eca-b248-26ffa998c5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:40.886712 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b410c78-5d16-4eca-b248-26ffa998c5b9 admin admin] PATH_INFO: `/v3/projects/20c305cbc2af42edac081133bb7b8915/users/0f48ba8af70a49d4a0a824487243c5ac/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:40.902988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b410c78-5d16-4eca-b248-26ffa998c5b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c305cbc2af42edac081133bb7b8915, user_id=0f48ba8af70a49d4a0a824487243c5ac, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:40.905739 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b410c78-5d16-4eca-b248-26ffa998c5b9 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:40.974570 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1420/2854] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:40 2022] PUT /identity/v3/projects/20c305cbc2af42edac081133bb7b8915/users/0f48ba8af70a49d4a0a824487243c5ac/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:40.979126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa1135-9cb3-4c09-92ae-17905aa039c3 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:40.990617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa1135-9cb3-4c09-92ae-17905aa039c3 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:41.050504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71155ab5-41a3-4804-a8e9-ab990e873ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:41.051188 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71155ab5-41a3-4804-a8e9-ab990e873ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:41.051539 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71155ab5-41a3-4804-a8e9-ab990e873ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:41.052019 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71155ab5-41a3-4804-a8e9-ab990e873ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:41.052683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71155ab5-41a3-4804-a8e9-ab990e873ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:41.054660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71155ab5-41a3-4804-a8e9-ab990e873ae5 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:41.065754 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1435/2855] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:41.070252 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b410c78-5d16-4eca-b248-26ffa998c5b9 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:41.081517 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b410c78-5d16-4eca-b248-26ffa998c5b9 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:41.086414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497c8b7-13cb-4266-a251-2e08e1c6f760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:41.087296 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497c8b7-13cb-4266-a251-2e08e1c6f760 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:41.087577 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497c8b7-13cb-4266-a251-2e08e1c6f760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:41.087852 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497c8b7-13cb-4266-a251-2e08e1c6f760 admin admin] PATH_INFO: `/v3/projects/20c305cbc2af42edac081133bb7b8915/users/0f48ba8af70a49d4a0a824487243c5ac/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:41.090746 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0497c8b7-13cb-4266-a251-2e08e1c6f760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c305cbc2af42edac081133bb7b8915, user_id=0f48ba8af70a49d4a0a824487243c5ac, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:41.093395 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0497c8b7-13cb-4266-a251-2e08e1c6f760 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:41.099925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:41.104274 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1421/2856] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:41 2022] PUT /identity/v3/projects/20c305cbc2af42edac081133bb7b8915/users/0f48ba8af70a49d4a0a824487243c5ac/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:41.115060 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b35bc9-64a4-4e08-91ad-72d32c8349f4 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:41.115348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b35bc9-64a4-4e08-91ad-72d32c8349f4 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:41.120728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b35bc9-64a4-4e08-91ad-72d32c8349f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:41.143058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-81b35bc9-64a4-4e08-91ad-72d32c8349f4 None None] MFA Rules not processed for user `0f48ba8af70a49d4a0a824487243c5ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:41.144256 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-81b35bc9-64a4-4e08-91ad-72d32c8349f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:41.211946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1436/2857] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:41.223402 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497c8b7-13cb-4266-a251-2e08e1c6f760 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:41.225146 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-0497c8b7-13cb-4266-a251-2e08e1c6f760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:41.297881 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497c8b7-13cb-4266-a251-2e08e1c6f760 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:41.301586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e6160-262a-4ebe-9085-77bbc5136b1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:41.302232 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e6160-262a-4ebe-9085-77bbc5136b1e service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:41.302517 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e6160-262a-4ebe-9085-77bbc5136b1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:41.302794 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e6160-262a-4ebe-9085-77bbc5136b1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:41.360900 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e6160-262a-4ebe-9085-77bbc5136b1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:41.362543 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e6160-262a-4ebe-9085-77bbc5136b1e service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:41.371966 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1422/2858] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:43.070951 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b35bc9-64a4-4e08-91ad-72d32c8349f4 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:43.073283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-81b35bc9-64a4-4e08-91ad-72d32c8349f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:43.094888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b35bc9-64a4-4e08-91ad-72d32c8349f4 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:43.100907 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abba30-d445-4029-8fde-540ad02a90af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:43.102084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abba30-d445-4029-8fde-540ad02a90af service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:43.102590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abba30-d445-4029-8fde-540ad02a90af service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:43.103103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abba30-d445-4029-8fde-540ad02a90af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:43.106666 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-92abba30-d445-4029-8fde-540ad02a90af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:43.180354 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abba30-d445-4029-8fde-540ad02a90af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:43.182130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abba30-d445-4029-8fde-540ad02a90af service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:43.193700 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1437/2859] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:44.683310 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e6160-262a-4ebe-9085-77bbc5136b1e service cinder] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:44.747844 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e6160-262a-4ebe-9085-77bbc5136b1e service cinder] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:44.753034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc65da1-9684-407c-bd26-7da3e69a4d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:44.753929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc65da1-9684-407c-bd26-7da3e69a4d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:44.754211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc65da1-9684-407c-bd26-7da3e69a4d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:44.754487 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc65da1-9684-407c-bd26-7da3e69a4d92 admin admin] PATH_INFO: `/v3/users/0aba9e39520a42d29989943938c3e0e0` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:44.755974 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc65da1-9684-407c-bd26-7da3e69a4d92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aba9e39520a42d29989943938c3e0e0)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:44.758023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc65da1-9684-407c-bd26-7da3e69a4d92 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:44.953839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:44.955650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9dc65da1-9684-407c-bd26-7da3e69a4d92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aba9e39520a42d29989943938c3e0e0'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:45.042665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9dc65da1-9684-407c-bd26-7da3e69a4d92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aba9e39520a42d29989943938c3e0e0'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:45.043851 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9dc65da1-9684-407c-bd26-7da3e69a4d92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aba9e39520a42d29989943938c3e0e0'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:45.044499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9dc65da1-9684-407c-bd26-7da3e69a4d92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aba9e39520a42d29989943938c3e0e0'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:45.057784 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9dc65da1-9684-407c-bd26-7da3e69a4d92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aba9e39520a42d29989943938c3e0e0'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:45.070760 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1423/2860] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:44 2022] DELETE /identity/v3/users/0aba9e39520a42d29989943938c3e0e0 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:45.075459 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d67a4-1e54-47dd-8378-40bbc323af78 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:45.075961 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d67a4-1e54-47dd-8378-40bbc323af78 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:45.076077 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d67a4-1e54-47dd-8378-40bbc323af78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:45.093560 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-3a7d67a4-1e54-47dd-8378-40bbc323af78 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:45.094800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-3a7d67a4-1e54-47dd-8378-40bbc323af78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:45.100343 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=100 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:45.159985 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1438/2861] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:45.171864 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc65da1-9684-407c-bd26-7da3e69a4d92 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:45.173502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9dc65da1-9684-407c-bd26-7da3e69a4d92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:45.236053 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc65da1-9684-407c-bd26-7da3e69a4d92 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:45.239990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812167a8-c46f-4706-b64b-d7b5115f910d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:45.240637 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812167a8-c46f-4706-b64b-d7b5115f910d service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:45.240925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812167a8-c46f-4706-b64b-d7b5115f910d service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:45.241233 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812167a8-c46f-4706-b64b-d7b5115f910d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:45.251536 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812167a8-c46f-4706-b64b-d7b5115f910d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:45.253202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812167a8-c46f-4706-b64b-d7b5115f910d service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:45.264212 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1424/2862] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:45.714370 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d67a4-1e54-47dd-8378-40bbc323af78 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:45.716536 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-3a7d67a4-1e54-47dd-8378-40bbc323af78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:45.730004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d67a4-1e54-47dd-8378-40bbc323af78 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:45.735380 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:45.736525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:45.736815 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:45.737184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab admin admin] PATH_INFO: `/v3/projects/31cb7b8ee8ac4e23961595d2d0336cff` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:45.738429 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=31cb7b8ee8ac4e23961595d2d0336cff)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:45.740486 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:45.850446 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31cb7b8ee8ac4e23961595d2d0336cff', 'request_id': 'req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:45.858716 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31cb7b8ee8ac4e23961595d2d0336cff', 'request_id': 'req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:45.865485 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31cb7b8ee8ac4e23961595d2d0336cff', 'request_id': 'req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:45.943690 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab admin admin] The token cache is being invalidate because project 31cb7b8ee8ac4e23961595d2d0336cff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:45.944034 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:45.944779 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:45.946732 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1439/2863] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:45 2022] DELETE /identity/v3/projects/31cb7b8ee8ac4e23961595d2d0336cff => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:45.953132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812167a8-c46f-4706-b64b-d7b5115f910d service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:45.964010 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812167a8-c46f-4706-b64b-d7b5115f910d service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:46.028071 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4703ac-4ad5-48d9-99e6-8434f2305760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:46.028889 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4703ac-4ad5-48d9-99e6-8434f2305760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:46.029211 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4703ac-4ad5-48d9-99e6-8434f2305760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:46.029514 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4703ac-4ad5-48d9-99e6-8434f2305760 admin admin] PATH_INFO: `/v3/users/0f48ba8af70a49d4a0a824487243c5ac` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:46.030792 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4703ac-4ad5-48d9-99e6-8434f2305760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f48ba8af70a49d4a0a824487243c5ac)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:46.032733 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4703ac-4ad5-48d9-99e6-8434f2305760 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:46.233598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9d4703ac-4ad5-48d9-99e6-8434f2305760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f48ba8af70a49d4a0a824487243c5ac'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:46.303095 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9d4703ac-4ad5-48d9-99e6-8434f2305760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f48ba8af70a49d4a0a824487243c5ac'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:46.303941 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9d4703ac-4ad5-48d9-99e6-8434f2305760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f48ba8af70a49d4a0a824487243c5ac'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:46.304626 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9d4703ac-4ad5-48d9-99e6-8434f2305760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f48ba8af70a49d4a0a824487243c5ac'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:46.317262 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:46.319756 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9d4703ac-4ad5-48d9-99e6-8434f2305760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f48ba8af70a49d4a0a824487243c5ac'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:46.333791 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1425/2864] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:03:45 2022] DELETE /identity/v3/users/0f48ba8af70a49d4a0a824487243c5ac => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:03:46.800462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:46.802048 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:46.816599 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289b457-a0b9-4618-97a8-61c6dd8cc0ab admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:46.880203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25458172-e0ba-4540-869c-7645758dc350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:46.881019 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25458172-e0ba-4540-869c-7645758dc350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:46.881311 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25458172-e0ba-4540-869c-7645758dc350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:46.881577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25458172-e0ba-4540-869c-7645758dc350 admin admin] PATH_INFO: `/v3/projects/20c305cbc2af42edac081133bb7b8915` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:46.882958 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25458172-e0ba-4540-869c-7645758dc350 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20c305cbc2af42edac081133bb7b8915)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:46.884965 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25458172-e0ba-4540-869c-7645758dc350 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:47.005067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:47.014672 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-25458172-e0ba-4540-869c-7645758dc350 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20c305cbc2af42edac081133bb7b8915', 'request_id': 'req-25458172-e0ba-4540-869c-7645758dc350'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:47.021628 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-25458172-e0ba-4540-869c-7645758dc350 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20c305cbc2af42edac081133bb7b8915', 'request_id': 'req-25458172-e0ba-4540-869c-7645758dc350'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:47.027211 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-25458172-e0ba-4540-869c-7645758dc350 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20c305cbc2af42edac081133bb7b8915', 'request_id': 'req-25458172-e0ba-4540-869c-7645758dc350'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:47.102320 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-25458172-e0ba-4540-869c-7645758dc350 admin admin] The token cache is being invalidate because project 20c305cbc2af42edac081133bb7b8915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:03:47.102615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-25458172-e0ba-4540-869c-7645758dc350 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:47.103284 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-25458172-e0ba-4540-869c-7645758dc350 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:03:47.109310 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1440/2865] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:03:46 2022] DELETE /identity/v3/projects/20c305cbc2af42edac081133bb7b8915 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:47.111018 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2123949-f83d-4e7f-b14e-ebf244e2e8f2 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:47.111837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2123949-f83d-4e7f-b14e-ebf244e2e8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:47.111837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2123949-f83d-4e7f-b14e-ebf244e2e8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:47.131203 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-e2123949-f83d-4e7f-b14e-ebf244e2e8f2 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:47.133011 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-e2123949-f83d-4e7f-b14e-ebf244e2e8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:47.238111 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1426/2866] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:47.251617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba35267-59ad-42c0-bae2-e74a7b16b797 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:47.251931 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba35267-59ad-42c0-bae2-e74a7b16b797 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:47.252230 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba35267-59ad-42c0-bae2-e74a7b16b797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:47.271301 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-1ba35267-59ad-42c0-bae2-e74a7b16b797 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:47.272544 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-1ba35267-59ad-42c0-bae2-e74a7b16b797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:47.281290 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1441/2867] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:03:47.285831 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2123949-f83d-4e7f-b14e-ebf244e2e8f2 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:47.296705 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2123949-f83d-4e7f-b14e-ebf244e2e8f2 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:47.301407 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efaa70-b103-4cd7-8c4a-ff9b71a3c45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:47.302072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efaa70-b103-4cd7-8c4a-ff9b71a3c45d admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:47.302355 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efaa70-b103-4cd7-8c4a-ff9b71a3c45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:47.302639 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efaa70-b103-4cd7-8c4a-ff9b71a3c45d admin admin] PATH_INFO: `/v3/domains` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:47.303376 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efaa70-b103-4cd7-8c4a-ff9b71a3c45d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:03:47.303693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efaa70-b103-4cd7-8c4a-ff9b71a3c45d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:47.305326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efaa70-b103-4cd7-8c4a-ff9b71a3c45d admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:47.317184 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1427/2868] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:03:47.321632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba35267-59ad-42c0-bae2-e74a7b16b797 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:47.331058 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba35267-59ad-42c0-bae2-e74a7b16b797 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:47.335803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555834b8-df19-42a1-8ca6-3ace463cf853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:47.336456 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555834b8-df19-42a1-8ca6-3ace463cf853 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:47.336743 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555834b8-df19-42a1-8ca6-3ace463cf853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:47.337056 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555834b8-df19-42a1-8ca6-3ace463cf853 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:47.337761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555834b8-df19-42a1-8ca6-3ace463cf853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:47.340274 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555834b8-df19-42a1-8ca6-3ace463cf853 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:47.414281 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1442/2869] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:03:47 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:47.417549 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efaa70-b103-4cd7-8c4a-ff9b71a3c45d admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:47.428462 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efaa70-b103-4cd7-8c4a-ff9b71a3c45d admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:47.487755 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6e1f0-f267-4e66-9117-7e1c8100f21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:47.488567 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6e1f0-f267-4e66-9117-7e1c8100f21d admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:47.488851 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6e1f0-f267-4e66-9117-7e1c8100f21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:47.489170 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6e1f0-f267-4e66-9117-7e1c8100f21d admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:47.489874 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6e1f0-f267-4e66-9117-7e1c8100f21d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:47.491851 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6e1f0-f267-4e66-9117-7e1c8100f21d admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:47.583215 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1428/2870] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:03:47 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:03:47.587752 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555834b8-df19-42a1-8ca6-3ace463cf853 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:47.599247 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555834b8-df19-42a1-8ca6-3ace463cf853 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:47.604074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc6f17-a118-47b8-b475-6306768abee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:47.604741 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc6f17-a118-47b8-b475-6306768abee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:47.605074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc6f17-a118-47b8-b475-6306768abee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:47.605396 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc6f17-a118-47b8-b475-6306768abee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:47.606355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc6f17-a118-47b8-b475-6306768abee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:47.607996 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc6f17-a118-47b8-b475-6306768abee2 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:47.620435 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1443/2871] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:47.625882 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6e1f0-f267-4e66-9117-7e1c8100f21d admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:47.637274 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6e1f0-f267-4e66-9117-7e1c8100f21d admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:47.642088 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c415e0-5647-4975-a7f2-523b1e74c36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:47.642763 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c415e0-5647-4975-a7f2-523b1e74c36c admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:47.643055 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c415e0-5647-4975-a7f2-523b1e74c36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:47.643330 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c415e0-5647-4975-a7f2-523b1e74c36c admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:47.644088 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c415e0-5647-4975-a7f2-523b1e74c36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:47.645732 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c415e0-5647-4975-a7f2-523b1e74c36c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:47.657674 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1429/2872] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:03:47.662084 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc6f17-a118-47b8-b475-6306768abee2 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:47.673024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc6f17-a118-47b8-b475-6306768abee2 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:47.677806 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7ef8e-9a5f-47b9-8486-1744b598aea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:47.678693 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7ef8e-9a5f-47b9-8486-1744b598aea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:47.678979 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7ef8e-9a5f-47b9-8486-1744b598aea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:47.679263 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7ef8e-9a5f-47b9-8486-1744b598aea4 admin admin] PATH_INFO: `/v3/projects/0531a4e00cc94812adc5e20d8fa98690/users/bbd0d428a40043f9a9e11ddc2f7902fb/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:47.695302 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7ef8e-9a5f-47b9-8486-1744b598aea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0531a4e00cc94812adc5e20d8fa98690, user_id=bbd0d428a40043f9a9e11ddc2f7902fb, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:47.698193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c7ef8e-9a5f-47b9-8486-1744b598aea4 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:47.781420 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1444/2873] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:03:47 2022] PUT /identity/v3/projects/0531a4e00cc94812adc5e20d8fa98690/users/bbd0d428a40043f9a9e11ddc2f7902fb/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:03:47.793688 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef70267-154e-4178-9178-880e7456f58d None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:47.794775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef70267-154e-4178-9178-880e7456f58d None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:47.794775 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef70267-154e-4178-9178-880e7456f58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:47.823837 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-aef70267-154e-4178-9178-880e7456f58d None None] MFA Rules not processed for user `bbd0d428a40043f9a9e11ddc2f7902fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:03:47.826006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-aef70267-154e-4178-9178-880e7456f58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:47.868534 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:47.894392 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1430/2874] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:03:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:03:47.905816 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7ef8e-9a5f-47b9-8486-1744b598aea4 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:03:47.907746 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-00c7ef8e-9a5f-47b9-8486-1744b598aea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:03:47.976586 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7ef8e-9a5f-47b9-8486-1744b598aea4 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:03:47.980574 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e74c1-1f9f-4bed-84b1-d149c2a7df58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:03:47.981299 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e74c1-1f9f-4bed-84b1-d149c2a7df58 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:03:47.981662 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e74c1-1f9f-4bed-84b1-d149c2a7df58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:03:47.981948 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e74c1-1f9f-4bed-84b1-d149c2a7df58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:03:47.992052 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e74c1-1f9f-4bed-84b1-d149c2a7df58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:03:47.995200 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e74c1-1f9f-4bed-84b1-d149c2a7df58 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:03:48.004362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1445/2875] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:03:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:03:57.957382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=9 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:03:57.999738 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=64,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:28.253872 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef70267-154e-4178-9178-880e7456f58d None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:28.255430 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-aef70267-154e-4178-9178-880e7456f58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:28.327397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef70267-154e-4178-9178-880e7456f58d None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:28.332719 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34893a05-608c-47b4-90ad-970acc6388d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:28.333639 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34893a05-608c-47b4-90ad-970acc6388d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:28.333920 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34893a05-608c-47b4-90ad-970acc6388d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:28.334186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34893a05-608c-47b4-90ad-970acc6388d1 admin admin] PATH_INFO: `/v3/users/39b3ab523c184f7f841fc0f959fa4072` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:28.335378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34893a05-608c-47b4-90ad-970acc6388d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39b3ab523c184f7f841fc0f959fa4072)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:28.338551 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34893a05-608c-47b4-90ad-970acc6388d1 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:28.589653 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-34893a05-608c-47b4-90ad-970acc6388d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39b3ab523c184f7f841fc0f959fa4072'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:28.683373 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-34893a05-608c-47b4-90ad-970acc6388d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39b3ab523c184f7f841fc0f959fa4072'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:28.684013 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-34893a05-608c-47b4-90ad-970acc6388d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39b3ab523c184f7f841fc0f959fa4072'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:28.684620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-34893a05-608c-47b4-90ad-970acc6388d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39b3ab523c184f7f841fc0f959fa4072'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:28.698649 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-34893a05-608c-47b4-90ad-970acc6388d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39b3ab523c184f7f841fc0f959fa4072'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:28.711209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1431/2876] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:04:28 2022] DELETE /identity/v3/users/39b3ab523c184f7f841fc0f959fa4072 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:28.716976 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4cb5b-5911-49c6-9f06-85e699d773ab None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:28.717473 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4cb5b-5911-49c6-9f06-85e699d773ab None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:28.717565 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4cb5b-5911-49c6-9f06-85e699d773ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:28.734824 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-7ee4cb5b-5911-49c6-9f06-85e699d773ab None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:28.736078 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7ee4cb5b-5911-49c6-9f06-85e699d773ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:28.796699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1446/2877] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:04:28.806936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34893a05-608c-47b4-90ad-970acc6388d1 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:28.808354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-34893a05-608c-47b4-90ad-970acc6388d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:28.880023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34893a05-608c-47b4-90ad-970acc6388d1 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:28.883640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2b41d-4804-49bc-a07e-7351d736518b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:28.884880 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2b41d-4804-49bc-a07e-7351d736518b service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:28.885208 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2b41d-4804-49bc-a07e-7351d736518b service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:28.885565 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2b41d-4804-49bc-a07e-7351d736518b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:28.894978 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2b41d-4804-49bc-a07e-7351d736518b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:28.896414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2b41d-4804-49bc-a07e-7351d736518b service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:28.906378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1432/2878] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:04:29.299345 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4cb5b-5911-49c6-9f06-85e699d773ab None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:29.300926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-7ee4cb5b-5911-49c6-9f06-85e699d773ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:29.314332 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4cb5b-5911-49c6-9f06-85e699d773ab None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:29.319411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848afe61-2332-4573-9d53-96a5db5caed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:29.320342 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848afe61-2332-4573-9d53-96a5db5caed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:29.320613 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848afe61-2332-4573-9d53-96a5db5caed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:29.320892 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848afe61-2332-4573-9d53-96a5db5caed7 admin admin] PATH_INFO: `/v3/projects/f0cbd9a794d6464da42ac3e163c4b1b9` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:29.322992 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848afe61-2332-4573-9d53-96a5db5caed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0cbd9a794d6464da42ac3e163c4b1b9)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:29.330408 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848afe61-2332-4573-9d53-96a5db5caed7 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:29.484647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-848afe61-2332-4573-9d53-96a5db5caed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0cbd9a794d6464da42ac3e163c4b1b9', 'request_id': 'req-848afe61-2332-4573-9d53-96a5db5caed7'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:29.491371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-848afe61-2332-4573-9d53-96a5db5caed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0cbd9a794d6464da42ac3e163c4b1b9', 'request_id': 'req-848afe61-2332-4573-9d53-96a5db5caed7'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:29.497537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-848afe61-2332-4573-9d53-96a5db5caed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0cbd9a794d6464da42ac3e163c4b1b9', 'request_id': 'req-848afe61-2332-4573-9d53-96a5db5caed7'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:29.586388 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-848afe61-2332-4573-9d53-96a5db5caed7 admin admin] The token cache is being invalidate because project f0cbd9a794d6464da42ac3e163c4b1b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:04:29.586578 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-848afe61-2332-4573-9d53-96a5db5caed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:29.587935 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-848afe61-2332-4573-9d53-96a5db5caed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:29.589579 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1447/2879] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:04:29 2022] DELETE /identity/v3/projects/f0cbd9a794d6464da42ac3e163c4b1b9 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:29.593693 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2b41d-4804-49bc-a07e-7351d736518b service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:29.604127 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2b41d-4804-49bc-a07e-7351d736518b service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:29.612231 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:29.662142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e8e20-51e0-45c6-a678-93553c7c71f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:29.663033 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e8e20-51e0-45c6-a678-93553c7c71f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:29.663313 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e8e20-51e0-45c6-a678-93553c7c71f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:29.663588 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e8e20-51e0-45c6-a678-93553c7c71f5 admin admin] PATH_INFO: `/v3/users/c58b7013fb374cec96bb6c64791a4369` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:29.665076 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e8e20-51e0-45c6-a678-93553c7c71f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c58b7013fb374cec96bb6c64791a4369)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:29.668832 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e8e20-51e0-45c6-a678-93553c7c71f5 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:29.871586 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-623e8e20-51e0-45c6-a678-93553c7c71f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c58b7013fb374cec96bb6c64791a4369'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:29.959575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-623e8e20-51e0-45c6-a678-93553c7c71f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c58b7013fb374cec96bb6c64791a4369'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:29.960185 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-623e8e20-51e0-45c6-a678-93553c7c71f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c58b7013fb374cec96bb6c64791a4369'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:29.960791 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-623e8e20-51e0-45c6-a678-93553c7c71f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c58b7013fb374cec96bb6c64791a4369'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:29.974662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-623e8e20-51e0-45c6-a678-93553c7c71f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c58b7013fb374cec96bb6c64791a4369'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:29.988108 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1433/2880] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:04:29 2022] DELETE /identity/v3/users/c58b7013fb374cec96bb6c64791a4369 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:30.445510 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848afe61-2332-4573-9d53-96a5db5caed7 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:30.447735 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-848afe61-2332-4573-9d53-96a5db5caed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:30.461362 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848afe61-2332-4573-9d53-96a5db5caed7 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:30.520207 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285479c-59e6-4ac3-a5bf-926c26a1d458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:30.521108 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285479c-59e6-4ac3-a5bf-926c26a1d458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:30.521389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285479c-59e6-4ac3-a5bf-926c26a1d458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:30.521663 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285479c-59e6-4ac3-a5bf-926c26a1d458 admin admin] PATH_INFO: `/v3/projects/6ed595bf80cb4678955ce81e9077c93b` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:30.522835 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285479c-59e6-4ac3-a5bf-926c26a1d458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ed595bf80cb4678955ce81e9077c93b)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:30.524911 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285479c-59e6-4ac3-a5bf-926c26a1d458 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:30.634346 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:30.652015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f285479c-59e6-4ac3-a5bf-926c26a1d458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ed595bf80cb4678955ce81e9077c93b', 'request_id': 'req-f285479c-59e6-4ac3-a5bf-926c26a1d458'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:30.658248 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f285479c-59e6-4ac3-a5bf-926c26a1d458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ed595bf80cb4678955ce81e9077c93b', 'request_id': 'req-f285479c-59e6-4ac3-a5bf-926c26a1d458'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:30.663770 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f285479c-59e6-4ac3-a5bf-926c26a1d458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ed595bf80cb4678955ce81e9077c93b', 'request_id': 'req-f285479c-59e6-4ac3-a5bf-926c26a1d458'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:30.743144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f285479c-59e6-4ac3-a5bf-926c26a1d458 admin admin] The token cache is being invalidate because project 6ed595bf80cb4678955ce81e9077c93b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:04:30.744155 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f285479c-59e6-4ac3-a5bf-926c26a1d458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:30.745004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-f285479c-59e6-4ac3-a5bf-926c26a1d458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:30.746622 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1448/2881] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:04:30 2022] DELETE /identity/v3/projects/6ed595bf80cb4678955ce81e9077c93b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:30.752270 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8407a-cfc3-47ef-8b0a-76ce2c45e5a1 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:30.752839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8407a-cfc3-47ef-8b0a-76ce2c45e5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:30.752839 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8407a-cfc3-47ef-8b0a-76ce2c45e5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:30.772668 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-94e8407a-cfc3-47ef-8b0a-76ce2c45e5a1 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:30.773959 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-94e8407a-cfc3-47ef-8b0a-76ce2c45e5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:30.848366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1434/2882] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:04:30.857886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61273db-4b2d-4354-95bf-d7b5bc1e1ff1 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:30.858275 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61273db-4b2d-4354-95bf-d7b5bc1e1ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:30.858636 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61273db-4b2d-4354-95bf-d7b5bc1e1ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:30.869863 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8407a-cfc3-47ef-8b0a-76ce2c45e5a1 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:30.871276 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-94e8407a-cfc3-47ef-8b0a-76ce2c45e5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:30.878941 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-c61273db-4b2d-4354-95bf-d7b5bc1e1ff1 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:30.880191 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c61273db-4b2d-4354-95bf-d7b5bc1e1ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:30.886502 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8407a-cfc3-47ef-8b0a-76ce2c45e5a1 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:30.889966 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1449/2883] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:04:30.891495 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b844f2-21ec-408f-8b79-9ad1bc48a49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:30.892373 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b844f2-21ec-408f-8b79-9ad1bc48a49b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:30.892656 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b844f2-21ec-408f-8b79-9ad1bc48a49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:30.892927 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b844f2-21ec-408f-8b79-9ad1bc48a49b admin admin] PATH_INFO: `/v3/users/bbd0d428a40043f9a9e11ddc2f7902fb` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:30.894294 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b844f2-21ec-408f-8b79-9ad1bc48a49b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbd0d428a40043f9a9e11ddc2f7902fb)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:30.894412 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61273db-4b2d-4354-95bf-d7b5bc1e1ff1 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:30.896250 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b844f2-21ec-408f-8b79-9ad1bc48a49b admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:30.906673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61273db-4b2d-4354-95bf-d7b5bc1e1ff1 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:30.911813 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66274b10-70f6-47a6-9c87-1151af0d10bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:30.912605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66274b10-70f6-47a6-9c87-1151af0d10bd admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:30.912890 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66274b10-70f6-47a6-9c87-1151af0d10bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:30.913204 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66274b10-70f6-47a6-9c87-1151af0d10bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:30.914004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66274b10-70f6-47a6-9c87-1151af0d10bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:04:30.914363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66274b10-70f6-47a6-9c87-1151af0d10bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:30.917069 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66274b10-70f6-47a6-9c87-1151af0d10bd admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:30.929279 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1450/2884] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 15 11:04:30.934403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66274b10-70f6-47a6-9c87-1151af0d10bd admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:30.944805 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66274b10-70f6-47a6-9c87-1151af0d10bd admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:30.950876 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2237e-a29c-4a0b-92ff-52b135022932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:30.951538 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2237e-a29c-4a0b-92ff-52b135022932 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:30.951828 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2237e-a29c-4a0b-92ff-52b135022932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:30.952176 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2237e-a29c-4a0b-92ff-52b135022932 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:30.952882 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2237e-a29c-4a0b-92ff-52b135022932 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:30.954838 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2237e-a29c-4a0b-92ff-52b135022932 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:30.983512 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:31.070303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1451/2885] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:04:30 2022] POST /identity/v3/projects => generated 389 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:04:31.074537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2237e-a29c-4a0b-92ff-52b135022932 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:31.090022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-25b844f2-21ec-408f-8b79-9ad1bc48a49b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbd0d428a40043f9a9e11ddc2f7902fb'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:31.139913 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2237e-a29c-4a0b-92ff-52b135022932 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:31.144781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aac002-2caa-4eb4-8cad-219e7212b3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:31.145494 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aac002-2caa-4eb4-8cad-219e7212b3a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:31.145780 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aac002-2caa-4eb4-8cad-219e7212b3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:31.146062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aac002-2caa-4eb4-8cad-219e7212b3a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:31.146768 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aac002-2caa-4eb4-8cad-219e7212b3a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:31.148724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aac002-2caa-4eb4-8cad-219e7212b3a7 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:31.178085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-25b844f2-21ec-408f-8b79-9ad1bc48a49b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbd0d428a40043f9a9e11ddc2f7902fb'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:31.178727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-25b844f2-21ec-408f-8b79-9ad1bc48a49b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbd0d428a40043f9a9e11ddc2f7902fb'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:31.179363 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-25b844f2-21ec-408f-8b79-9ad1bc48a49b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbd0d428a40043f9a9e11ddc2f7902fb'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:31.195533 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-25b844f2-21ec-408f-8b79-9ad1bc48a49b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbd0d428a40043f9a9e11ddc2f7902fb'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:31.210216 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1435/2886] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:04:30 2022] DELETE /identity/v3/users/bbd0d428a40043f9a9e11ddc2f7902fb => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:04:31.214822 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab8546-d30f-4298-8d7d-a1d8e451f868 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:31.215118 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab8546-d30f-4298-8d7d-a1d8e451f868 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:31.215475 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab8546-d30f-4298-8d7d-a1d8e451f868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:31.233472 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-64ab8546-d30f-4298-8d7d-a1d8e451f868 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:31.234753 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-64ab8546-d30f-4298-8d7d-a1d8e451f868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:31.241428 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1452/2887] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:04:31 2022] POST /identity/v3/users => generated 295 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:04:31.246067 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aac002-2caa-4eb4-8cad-219e7212b3a7 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:31.247991 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-96aac002-2caa-4eb4-8cad-219e7212b3a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:31.264307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aac002-2caa-4eb4-8cad-219e7212b3a7 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:31.311301 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1436/2888] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:04:31.325357 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab8546-d30f-4298-8d7d-a1d8e451f868 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:31.326694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-64ab8546-d30f-4298-8d7d-a1d8e451f868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:31.329369 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cc784-33b9-4f23-85dd-95a6e9194eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:31.330083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cc784-33b9-4f23-85dd-95a6e9194eeb admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:31.330371 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cc784-33b9-4f23-85dd-95a6e9194eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:31.330654 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cc784-33b9-4f23-85dd-95a6e9194eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:31.331439 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092cc784-33b9-4f23-85dd-95a6e9194eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:31.334138 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092cc784-33b9-4f23-85dd-95a6e9194eeb admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:31.337453 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:31.347393 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1453/2889] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:04:31.351718 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cc784-33b9-4f23-85dd-95a6e9194eeb admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:31.363295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cc784-33b9-4f23-85dd-95a6e9194eeb admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:31.368203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcfa91-7801-488b-832b-193076da33dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:31.369411 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcfa91-7801-488b-832b-193076da33dd admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:31.369698 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcfa91-7801-488b-832b-193076da33dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:31.370015 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebcfa91-7801-488b-832b-193076da33dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:31.370758 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcfa91-7801-488b-832b-193076da33dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:31.372379 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebcfa91-7801-488b-832b-193076da33dd admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:31.383437 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1454/2890] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:04:31.389412 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcfa91-7801-488b-832b-193076da33dd admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:31.391982 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab8546-d30f-4298-8d7d-a1d8e451f868 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:31.395913 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392cdfe0-216f-4219-b5fb-e372f571ce02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:31.396572 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cdfe0-216f-4219-b5fb-e372f571ce02 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:31.396857 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cdfe0-216f-4219-b5fb-e372f571ce02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:31.397163 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cdfe0-216f-4219-b5fb-e372f571ce02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:31.401363 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebcfa91-7801-488b-832b-193076da33dd admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:31.407575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392cdfe0-216f-4219-b5fb-e372f571ce02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:31.407846 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275552db-e570-46a5-9a28-d0370e2f55a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:31.408821 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275552db-e570-46a5-9a28-d0370e2f55a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:31.409053 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392cdfe0-216f-4219-b5fb-e372f571ce02 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:31.409153 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275552db-e570-46a5-9a28-d0370e2f55a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:31.409403 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275552db-e570-46a5-9a28-d0370e2f55a7 admin admin] PATH_INFO: `/v3/projects/2a18ac2a5c244fb3ae66d30120ef1a24/users/8fb29eedde52489f81699d22471af239/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:31.420654 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1437/2891] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:04:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 15 11:04:31.436142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275552db-e570-46a5-9a28-d0370e2f55a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a18ac2a5c244fb3ae66d30120ef1a24, user_id=8fb29eedde52489f81699d22471af239, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:31.438856 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275552db-e570-46a5-9a28-d0370e2f55a7 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:31.547498 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1455/2892] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:04:31 2022] PUT /identity/v3/projects/2a18ac2a5c244fb3ae66d30120ef1a24/users/8fb29eedde52489f81699d22471af239/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:31.559745 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdcbf6-50ac-42e4-9be2-2d00177ff6b9 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:31.560040 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdcbf6-50ac-42e4-9be2-2d00177ff6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:31.560321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdcbf6-50ac-42e4-9be2-2d00177ff6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:31.586282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b5fdcbf6-50ac-42e4-9be2-2d00177ff6b9 None None] MFA Rules not processed for user `8fb29eedde52489f81699d22471af239`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:31.587827 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b5fdcbf6-50ac-42e4-9be2-2d00177ff6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:31.624740 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:31.657341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1438/2893] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:04:31.661889 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275552db-e570-46a5-9a28-d0370e2f55a7 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:31.755106 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275552db-e570-46a5-9a28-d0370e2f55a7 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:31.760340 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a6a1d-9c4b-450d-8cbb-dd00903bb488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:31.761196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a6a1d-9c4b-450d-8cbb-dd00903bb488 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:31.761473 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a6a1d-9c4b-450d-8cbb-dd00903bb488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:31.762203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a6a1d-9c4b-450d-8cbb-dd00903bb488 admin admin] PATH_INFO: `/v3/projects` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:31.762452 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a6a1d-9c4b-450d-8cbb-dd00903bb488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:31.764440 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a6a1d-9c4b-450d-8cbb-dd00903bb488 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:31.879520 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1456/2894] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:04:31 2022] POST /identity/v3/projects => generated 389 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:04:31.883657 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fdcbf6-50ac-42e4-9be2-2d00177ff6b9 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:31.896473 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fdcbf6-50ac-42e4-9be2-2d00177ff6b9 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:31.912431 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a6a1d-9c4b-450d-8cbb-dd00903bb488 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:31.914193 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6d5a6a1d-9c4b-450d-8cbb-dd00903bb488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:31.959645 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9a247-1f10-4058-9f5c-13e380c8c4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:31.960468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9a247-1f10-4058-9f5c-13e380c8c4cd admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:31.960859 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9a247-1f10-4058-9f5c-13e380c8c4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:31.961281 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9a247-1f10-4058-9f5c-13e380c8c4cd admin admin] PATH_INFO: `/v3/users` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:31.962080 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe9a247-1f10-4058-9f5c-13e380c8c4cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:31.964651 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe9a247-1f10-4058-9f5c-13e380c8c4cd admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:31.988670 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a6a1d-9c4b-450d-8cbb-dd00903bb488 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:31.993436 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:31.994293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:31.994575 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:31.995008 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe admin admin] PATH_INFO: `/v3/projects/0531a4e00cc94812adc5e20d8fa98690` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:31.996203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe admin admin] RBAC: Authorizing `identity:delete_project(project_id=0531a4e00cc94812adc5e20d8fa98690)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:31.998225 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:32.046423 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1439/2895] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:04:31 2022] POST /identity/v3/users => generated 301 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:04:32.050220 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9a247-1f10-4058-9f5c-13e380c8c4cd admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:32.060291 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe9a247-1f10-4058-9f5c-13e380c8c4cd admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:32.065591 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e358b4-3417-4bdd-a4bf-54a4b23c7412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:32.066305 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e358b4-3417-4bdd-a4bf-54a4b23c7412 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:32.066594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e358b4-3417-4bdd-a4bf-54a4b23c7412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:32.074907 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e358b4-3417-4bdd-a4bf-54a4b23c7412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:32.074907 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e358b4-3417-4bdd-a4bf-54a4b23c7412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:32.074907 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e358b4-3417-4bdd-a4bf-54a4b23c7412 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:32.079988 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1440/2896] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:04:32.083996 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e358b4-3417-4bdd-a4bf-54a4b23c7412 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:32.094650 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e358b4-3417-4bdd-a4bf-54a4b23c7412 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:32.111615 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0531a4e00cc94812adc5e20d8fa98690', 'request_id': 'req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:32.112699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:32.118781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0531a4e00cc94812adc5e20d8fa98690', 'request_id': 'req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:32.124638 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0531a4e00cc94812adc5e20d8fa98690', 'request_id': 'req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:32.154415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870282b-1a54-43d3-a818-fad5e370d887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:32.155229 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870282b-1a54-43d3-a818-fad5e370d887 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:32.155512 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870282b-1a54-43d3-a818-fad5e370d887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:32.155790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870282b-1a54-43d3-a818-fad5e370d887 admin admin] PATH_INFO: `/v3/projects/e1e8f1a3126b450fa8bfad93b1335889/users/ef62b7d423ed485ea48159ce5e4ec324/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:32.171136 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870282b-1a54-43d3-a818-fad5e370d887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1e8f1a3126b450fa8bfad93b1335889, user_id=ef62b7d423ed485ea48159ce5e4ec324, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:32.173894 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870282b-1a54-43d3-a818-fad5e370d887 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:32.187886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe admin admin] The token cache is being invalidate because project 0531a4e00cc94812adc5e20d8fa98690 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:04:32.188252 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:32.189014 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:32.191896 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1457/2897] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:04:31 2022] DELETE /identity/v3/projects/0531a4e00cc94812adc5e20d8fa98690 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:04:32.273828 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1441/2898] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:04:32 2022] PUT /identity/v3/projects/e1e8f1a3126b450fa8bfad93b1335889/users/ef62b7d423ed485ea48159ce5e4ec324/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:04:32.278767 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:32.281063 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:32.296039 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740f9e0-dc29-4dce-9651-fb2dbe6f7efe admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:32.361021 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e83bd-b7e4-4425-8df5-61b2e9e7c27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:32.361725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e83bd-b7e4-4425-8df5-61b2e9e7c27d admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:32.362007 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e83bd-b7e4-4425-8df5-61b2e9e7c27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:32.362282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e83bd-b7e4-4425-8df5-61b2e9e7c27d admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:32.363281 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e83bd-b7e4-4425-8df5-61b2e9e7c27d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:32.364916 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e83bd-b7e4-4425-8df5-61b2e9e7c27d admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:32.376531 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1458/2899] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:04:32.380888 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870282b-1a54-43d3-a818-fad5e370d887 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:32.392645 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870282b-1a54-43d3-a818-fad5e370d887 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:32.397812 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27894616-cd8b-4e87-be02-8fb1def9cd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:32.398769 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27894616-cd8b-4e87-be02-8fb1def9cd66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:32.399037 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27894616-cd8b-4e87-be02-8fb1def9cd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:32.399321 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27894616-cd8b-4e87-be02-8fb1def9cd66 admin admin] PATH_INFO: `/v3/projects/e1e8f1a3126b450fa8bfad93b1335889/users/ef62b7d423ed485ea48159ce5e4ec324/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:32.402002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27894616-cd8b-4e87-be02-8fb1def9cd66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1e8f1a3126b450fa8bfad93b1335889, user_id=ef62b7d423ed485ea48159ce5e4ec324, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:32.404790 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27894616-cd8b-4e87-be02-8fb1def9cd66 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:32.416106 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1442/2900] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:04:32 2022] PUT /identity/v3/projects/e1e8f1a3126b450fa8bfad93b1335889/users/ef62b7d423ed485ea48159ce5e4ec324/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:32.476355 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc493f8-2e5c-412f-955e-838122a2fe22 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:32.476654 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc493f8-2e5c-412f-955e-838122a2fe22 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:32.477056 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc493f8-2e5c-412f-955e-838122a2fe22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:32.504521 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-1cc493f8-2e5c-412f-955e-838122a2fe22 None None] MFA Rules not processed for user `ef62b7d423ed485ea48159ce5e4ec324`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:32.505797 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-1cc493f8-2e5c-412f-955e-838122a2fe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:32.575322 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1459/2901] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:04:32.593455 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27894616-cd8b-4e87-be02-8fb1def9cd66 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:32.595298 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-27894616-cd8b-4e87-be02-8fb1def9cd66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:32.625566 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:32.654894 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27894616-cd8b-4e87-be02-8fb1def9cd66 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:32.658919 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632f487-9c1e-44d7-8945-a564b51f55d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:32.659564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632f487-9c1e-44d7-8945-a564b51f55d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:32.659854 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632f487-9c1e-44d7-8945-a564b51f55d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:32.660130 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632f487-9c1e-44d7-8945-a564b51f55d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:32.670085 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632f487-9c1e-44d7-8945-a564b51f55d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:32.671515 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632f487-9c1e-44d7-8945-a564b51f55d2 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:32.681006 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1443/2902] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:04:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:04:32.819645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc493f8-2e5c-412f-955e-838122a2fe22 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:32.885184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc493f8-2e5c-412f-955e-838122a2fe22 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:32.890885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76c601-a9aa-42b9-9302-ae8e7cfd11d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:32.891763 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76c601-a9aa-42b9-9302-ae8e7cfd11d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:32.892045 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76c601-a9aa-42b9-9302-ae8e7cfd11d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:32.892322 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76c601-a9aa-42b9-9302-ae8e7cfd11d2 admin admin] PATH_INFO: `/v3/users/8fb29eedde52489f81699d22471af239` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:32.893531 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76c601-a9aa-42b9-9302-ae8e7cfd11d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fb29eedde52489f81699d22471af239)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:32.895454 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76c601-a9aa-42b9-9302-ae8e7cfd11d2 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:32.897088 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:33.074885 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6c76c601-a9aa-42b9-9302-ae8e7cfd11d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fb29eedde52489f81699d22471af239'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:33.086699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6c76c601-a9aa-42b9-9302-ae8e7cfd11d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fb29eedde52489f81699d22471af239'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:33.159461 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6c76c601-a9aa-42b9-9302-ae8e7cfd11d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fb29eedde52489f81699d22471af239'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:33.166791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6c76c601-a9aa-42b9-9302-ae8e7cfd11d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fb29eedde52489f81699d22471af239'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:33.167455 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-6c76c601-a9aa-42b9-9302-ae8e7cfd11d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fb29eedde52489f81699d22471af239'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:33.173519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1460/2903] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:04:32 2022] DELETE /identity/v3/users/8fb29eedde52489f81699d22471af239 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:04:33.178129 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb760f66-4cf7-45a8-92c7-bdcbd414fb85 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:33.178620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb760f66-4cf7-45a8-92c7-bdcbd414fb85 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:33.180024 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb760f66-4cf7-45a8-92c7-bdcbd414fb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:33.197315 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-bb760f66-4cf7-45a8-92c7-bdcbd414fb85 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:33.199245 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-bb760f66-4cf7-45a8-92c7-bdcbd414fb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:33.257548 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1444/2904] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 15 11:04:33.268297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76c601-a9aa-42b9-9302-ae8e7cfd11d2 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:33.269713 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-6c76c601-a9aa-42b9-9302-ae8e7cfd11d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:33.328129 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76c601-a9aa-42b9-9302-ae8e7cfd11d2 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:33.331626 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9c559-a13b-4d5d-90ee-04185ddc2cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:33.332329 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9c559-a13b-4d5d-90ee-04185ddc2cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:33.332614 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9c559-a13b-4d5d-90ee-04185ddc2cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:33.332891 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9c559-a13b-4d5d-90ee-04185ddc2cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:33.342167 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9c559-a13b-4d5d-90ee-04185ddc2cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:33.343605 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9c559-a13b-4d5d-90ee-04185ddc2cb8 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:33.353348 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1461/2905] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:04:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:04:33.761167 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb760f66-4cf7-45a8-92c7-bdcbd414fb85 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:33.762736 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-bb760f66-4cf7-45a8-92c7-bdcbd414fb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:33.775528 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb760f66-4cf7-45a8-92c7-bdcbd414fb85 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:33.781096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:33.781976 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:33.782260 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:33.782573 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7 admin admin] PATH_INFO: `/v3/projects/2a18ac2a5c244fb3ae66d30120ef1a24` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:33.784105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a18ac2a5c244fb3ae66d30120ef1a24)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:33.785944 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:33.902742 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a18ac2a5c244fb3ae66d30120ef1a24', 'request_id': 'req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:33.909366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a18ac2a5c244fb3ae66d30120ef1a24', 'request_id': 'req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:33.992014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a18ac2a5c244fb3ae66d30120ef1a24', 'request_id': 'req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:34.004079 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7 admin admin] The token cache is being invalidate because project 2a18ac2a5c244fb3ae66d30120ef1a24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:04:34.007797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:34.007797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:34.007797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1445/2906] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:04:33 2022] DELETE /identity/v3/projects/2a18ac2a5c244fb3ae66d30120ef1a24 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:34.010803 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9c559-a13b-4d5d-90ee-04185ddc2cb8 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:34.020680 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9c559-a13b-4d5d-90ee-04185ddc2cb8 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:34.076936 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47334862-ed85-4abf-8b46-29006c5a3b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:34.077869 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47334862-ed85-4abf-8b46-29006c5a3b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:34.078156 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47334862-ed85-4abf-8b46-29006c5a3b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:34.078433 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47334862-ed85-4abf-8b46-29006c5a3b90 admin admin] PATH_INFO: `/v3/users/ef62b7d423ed485ea48159ce5e4ec324` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:34.079646 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47334862-ed85-4abf-8b46-29006c5a3b90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef62b7d423ed485ea48159ce5e4ec324)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:34.081626 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47334862-ed85-4abf-8b46-29006c5a3b90 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:34.104859 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:34.276623 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-47334862-ed85-4abf-8b46-29006c5a3b90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef62b7d423ed485ea48159ce5e4ec324'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:34.289126 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-47334862-ed85-4abf-8b46-29006c5a3b90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef62b7d423ed485ea48159ce5e4ec324'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:34.382731 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-47334862-ed85-4abf-8b46-29006c5a3b90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef62b7d423ed485ea48159ce5e4ec324'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:34.391291 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-47334862-ed85-4abf-8b46-29006c5a3b90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef62b7d423ed485ea48159ce5e4ec324'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:34.391937 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-47334862-ed85-4abf-8b46-29006c5a3b90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef62b7d423ed485ea48159ce5e4ec324'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:34.397899 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1462/2907] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:04:34 2022] DELETE /identity/v3/users/ef62b7d423ed485ea48159ce5e4ec324 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:34.844472 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:34.846282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:34.865448 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46dcc6-7c66-4414-bd2d-dda1ac4beda7 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:34.914391 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:34.924266 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e37589-f1d1-4711-9f98-cd5e21b69d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:34.925175 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e37589-f1d1-4711-9f98-cd5e21b69d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:34.925458 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e37589-f1d1-4711-9f98-cd5e21b69d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:34.925742 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e37589-f1d1-4711-9f98-cd5e21b69d59 admin admin] PATH_INFO: `/v3/projects/e1e8f1a3126b450fa8bfad93b1335889` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:34.926930 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e37589-f1d1-4711-9f98-cd5e21b69d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1e8f1a3126b450fa8bfad93b1335889)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:34.928835 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e37589-f1d1-4711-9f98-cd5e21b69d59 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:35.032181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f5e37589-f1d1-4711-9f98-cd5e21b69d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1e8f1a3126b450fa8bfad93b1335889', 'request_id': 'req-f5e37589-f1d1-4711-9f98-cd5e21b69d59'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:35.038443 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f5e37589-f1d1-4711-9f98-cd5e21b69d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1e8f1a3126b450fa8bfad93b1335889', 'request_id': 'req-f5e37589-f1d1-4711-9f98-cd5e21b69d59'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:35.100509 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f5e37589-f1d1-4711-9f98-cd5e21b69d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1e8f1a3126b450fa8bfad93b1335889', 'request_id': 'req-f5e37589-f1d1-4711-9f98-cd5e21b69d59'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:35.112132 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f5e37589-f1d1-4711-9f98-cd5e21b69d59 admin admin] The token cache is being invalidate because project e1e8f1a3126b450fa8bfad93b1335889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:04:35.112427 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f5e37589-f1d1-4711-9f98-cd5e21b69d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:35.113346 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-f5e37589-f1d1-4711-9f98-cd5e21b69d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:35.114888 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1446/2908] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:04:34 2022] DELETE /identity/v3/projects/e1e8f1a3126b450fa8bfad93b1335889 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:35.120361 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55794a1-8d10-483b-993d-48d1676f5b2a None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:35.120852 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55794a1-8d10-483b-993d-48d1676f5b2a None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:35.120945 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55794a1-8d10-483b-993d-48d1676f5b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:35.141065 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-c55794a1-8d10-483b-993d-48d1676f5b2a None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:35.142318 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c55794a1-8d10-483b-993d-48d1676f5b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:35.205218 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1463/2909] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:04:35.212846 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae047030-98ed-4679-8b30-c9df17b585c1 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:35.213186 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae047030-98ed-4679-8b30-c9df17b585c1 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:35.213504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae047030-98ed-4679-8b30-c9df17b585c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:35.232488 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-ae047030-98ed-4679-8b30-c9df17b585c1 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:35.233751 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-ae047030-98ed-4679-8b30-c9df17b585c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:35.241960 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1447/2910] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:04:35.247100 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55794a1-8d10-483b-993d-48d1676f5b2a None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:35.256680 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55794a1-8d10-483b-993d-48d1676f5b2a None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:35.261778 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7f54b-4c78-4355-a305-e1cd0dac26aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:35.262427 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7f54b-4c78-4355-a305-e1cd0dac26aa admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:35.262710 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7f54b-4c78-4355-a305-e1cd0dac26aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:35.262987 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7f54b-4c78-4355-a305-e1cd0dac26aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:35.263717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7f54b-4c78-4355-a305-e1cd0dac26aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:04:35.264026 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7f54b-4c78-4355-a305-e1cd0dac26aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:35.265654 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7f54b-4c78-4355-a305-e1cd0dac26aa admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:35.277570 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1464/2911] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:04:35.281886 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae047030-98ed-4679-8b30-c9df17b585c1 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:35.291180 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae047030-98ed-4679-8b30-c9df17b585c1 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:35.296568 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bd760-ecb7-4827-a226-c2d0facf5ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:35.297259 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bd760-ecb7-4827-a226-c2d0facf5ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:35.297546 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bd760-ecb7-4827-a226-c2d0facf5ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:35.297829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bd760-ecb7-4827-a226-c2d0facf5ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:35.298537 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bd760-ecb7-4827-a226-c2d0facf5ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:35.300477 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bd760-ecb7-4827-a226-c2d0facf5ce6 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:35.391594 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1448/2912] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:04:35 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:04:35.396035 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7f54b-4c78-4355-a305-e1cd0dac26aa admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:35.406397 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7f54b-4c78-4355-a305-e1cd0dac26aa admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:35.464064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd6a26-9dd4-48a1-be1f-a848c6e18be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:35.464726 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd6a26-9dd4-48a1-be1f-a848c6e18be2 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:35.465049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd6a26-9dd4-48a1-be1f-a848c6e18be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:35.465335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd6a26-9dd4-48a1-be1f-a848c6e18be2 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:35.466020 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd6a26-9dd4-48a1-be1f-a848c6e18be2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:35.467983 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd6a26-9dd4-48a1-be1f-a848c6e18be2 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:35.561177 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1465/2913] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:04:35 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:04:35.565366 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bd760-ecb7-4827-a226-c2d0facf5ce6 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:35.576537 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bd760-ecb7-4827-a226-c2d0facf5ce6 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:35.581484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d40b1c-8f6a-46c9-acc2-2f0659da225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:35.582190 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d40b1c-8f6a-46c9-acc2-2f0659da225c admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:35.582478 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d40b1c-8f6a-46c9-acc2-2f0659da225c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:35.582761 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d40b1c-8f6a-46c9-acc2-2f0659da225c admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:35.583516 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d40b1c-8f6a-46c9-acc2-2f0659da225c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:35.585142 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d40b1c-8f6a-46c9-acc2-2f0659da225c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:35.596667 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1449/2914] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:04:35.600780 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd6a26-9dd4-48a1-be1f-a848c6e18be2 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:35.612075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd6a26-9dd4-48a1-be1f-a848c6e18be2 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:35.617056 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0d07f-ca29-4c3c-9b02-9f297e68c05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:35.617715 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0d07f-ca29-4c3c-9b02-9f297e68c05b admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:35.617999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0d07f-ca29-4c3c-9b02-9f297e68c05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:35.618275 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0d07f-ca29-4c3c-9b02-9f297e68c05b admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:35.619002 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0d07f-ca29-4c3c-9b02-9f297e68c05b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:35.620939 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0d07f-ca29-4c3c-9b02-9f297e68c05b admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:35.630763 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1466/2915] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:04:35.634876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d40b1c-8f6a-46c9-acc2-2f0659da225c admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:35.645147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d40b1c-8f6a-46c9-acc2-2f0659da225c admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:35.649730 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3aef08-deaa-454a-94d9-4ae670505bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:35.650611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3aef08-deaa-454a-94d9-4ae670505bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:35.651430 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3aef08-deaa-454a-94d9-4ae670505bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:35.651713 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3aef08-deaa-454a-94d9-4ae670505bd8 admin admin] PATH_INFO: `/v3/projects/3781491a821c4852aa9362606529fff4/users/26685d1290dd45be87a23ed2a45bb78e/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:35.666415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3aef08-deaa-454a-94d9-4ae670505bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3781491a821c4852aa9362606529fff4, user_id=26685d1290dd45be87a23ed2a45bb78e, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:35.669891 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3aef08-deaa-454a-94d9-4ae670505bd8 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:35.752824 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1450/2916] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:04:35 2022] PUT /identity/v3/projects/3781491a821c4852aa9362606529fff4/users/26685d1290dd45be87a23ed2a45bb78e/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:35.765380 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54db85e-1447-442f-a3d9-fe132b090e43 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:35.765673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54db85e-1447-442f-a3d9-fe132b090e43 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:35.765958 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54db85e-1447-442f-a3d9-fe132b090e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:35.780693 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:35.796122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-a54db85e-1447-442f-a3d9-fe132b090e43 None None] MFA Rules not processed for user `26685d1290dd45be87a23ed2a45bb78e`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:35.797381 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-a54db85e-1447-442f-a3d9-fe132b090e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:35.862384 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1467/2917] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:04:35.867103 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3aef08-deaa-454a-94d9-4ae670505bd8 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:35.931575 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3aef08-deaa-454a-94d9-4ae670505bd8 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:35.936948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10719fc9-6e2d-4180-959d-049960f13177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:35.937664 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10719fc9-6e2d-4180-959d-049960f13177 admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:35.937956 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10719fc9-6e2d-4180-959d-049960f13177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:35.938243 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10719fc9-6e2d-4180-959d-049960f13177 admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:35.939352 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10719fc9-6e2d-4180-959d-049960f13177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:35.940949 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10719fc9-6e2d-4180-959d-049960f13177 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:36.031019 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1451/2918] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:04:35 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:04:36.035321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54db85e-1447-442f-a3d9-fe132b090e43 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:36.046761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54db85e-1447-442f-a3d9-fe132b090e43 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:36.103933 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2846102-6566-43f4-bd52-c22248842946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:36.104660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2846102-6566-43f4-bd52-c22248842946 admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:36.104946 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2846102-6566-43f4-bd52-c22248842946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:36.105253 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2846102-6566-43f4-bd52-c22248842946 admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:36.105934 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2846102-6566-43f4-bd52-c22248842946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:36.107853 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2846102-6566-43f4-bd52-c22248842946 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:36.233343 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1468/2919] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:04:36 2022] POST /identity/v3/users => generated 299 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:04:36.237922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10719fc9-6e2d-4180-959d-049960f13177 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:36.243803 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:36.249520 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10719fc9-6e2d-4180-959d-049960f13177 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:36.254217 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31498f00-9c2a-4e13-bde2-5241a75834f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:36.254962 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31498f00-9c2a-4e13-bde2-5241a75834f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:36.255194 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31498f00-9c2a-4e13-bde2-5241a75834f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:36.255474 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31498f00-9c2a-4e13-bde2-5241a75834f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:36.256209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31498f00-9c2a-4e13-bde2-5241a75834f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:36.257917 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31498f00-9c2a-4e13-bde2-5241a75834f1 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:36.269031 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1452/2920] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:04:36.273238 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2846102-6566-43f4-bd52-c22248842946 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:36.284724 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2846102-6566-43f4-bd52-c22248842946 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:36.289339 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb1b71-22d5-4ba3-b8bd-3a4431778922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:36.290272 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb1b71-22d5-4ba3-b8bd-3a4431778922 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:36.290594 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb1b71-22d5-4ba3-b8bd-3a4431778922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:36.290870 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb1b71-22d5-4ba3-b8bd-3a4431778922 admin admin] PATH_INFO: `/v3/projects/1c06790433e249f8af83ff1f863af532/users/bdfc0057a46f475892b52ab55619c7e4/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:36.306699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb1b71-22d5-4ba3-b8bd-3a4431778922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c06790433e249f8af83ff1f863af532, user_id=bdfc0057a46f475892b52ab55619c7e4, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:36.310632 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb1b71-22d5-4ba3-b8bd-3a4431778922 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:36.421838 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1469/2921] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:04:36 2022] PUT /identity/v3/projects/1c06790433e249f8af83ff1f863af532/users/bdfc0057a46f475892b52ab55619c7e4/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:36.425865 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31498f00-9c2a-4e13-bde2-5241a75834f1 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:36.436985 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31498f00-9c2a-4e13-bde2-5241a75834f1 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:36.493484 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf9abd-c824-4ab1-b529-58ff4d8d5432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:36.494066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf9abd-c824-4ab1-b529-58ff4d8d5432 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:36.494350 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf9abd-c824-4ab1-b529-58ff4d8d5432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:36.494625 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf9abd-c824-4ab1-b529-58ff4d8d5432 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:36.495417 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf9abd-c824-4ab1-b529-58ff4d8d5432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:36.497990 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf9abd-c824-4ab1-b529-58ff4d8d5432 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:36.508656 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1453/2922] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:04:36.514073 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb1b71-22d5-4ba3-b8bd-3a4431778922 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:36.524889 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb1b71-22d5-4ba3-b8bd-3a4431778922 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:36.530514 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7e97d-9167-4383-bc67-93b7993246c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:36.531412 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7e97d-9167-4383-bc67-93b7993246c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:36.531694 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7e97d-9167-4383-bc67-93b7993246c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:36.531970 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7e97d-9167-4383-bc67-93b7993246c6 admin admin] PATH_INFO: `/v3/projects/1c06790433e249f8af83ff1f863af532/users/bdfc0057a46f475892b52ab55619c7e4/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:36.533973 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7e97d-9167-4383-bc67-93b7993246c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c06790433e249f8af83ff1f863af532, user_id=bdfc0057a46f475892b52ab55619c7e4, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:36.536708 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7e97d-9167-4383-bc67-93b7993246c6 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:36.548419 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1470/2923] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:04:36 2022] PUT /identity/v3/projects/1c06790433e249f8af83ff1f863af532/users/bdfc0057a46f475892b52ab55619c7e4/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:36.562625 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8010c4-51c6-4899-b76c-335f0e90ed22 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:36.562918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8010c4-51c6-4899-b76c-335f0e90ed22 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:36.563197 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8010c4-51c6-4899-b76c-335f0e90ed22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:36.591354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-cd8010c4-51c6-4899-b76c-335f0e90ed22 None None] MFA Rules not processed for user `bdfc0057a46f475892b52ab55619c7e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:36.592571 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-cd8010c4-51c6-4899-b76c-335f0e90ed22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:36.671464 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1454/2924] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:04:36.682660 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7e97d-9167-4383-bc67-93b7993246c6 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:36.686055 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-dbf7e97d-9167-4383-bc67-93b7993246c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:36.743860 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:36.748468 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7e97d-9167-4383-bc67-93b7993246c6 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:36.752789 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2089780-5338-46cb-90a9-30c794e93e77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:36.753478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2089780-5338-46cb-90a9-30c794e93e77 service cinder] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:36.753792 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2089780-5338-46cb-90a9-30c794e93e77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:36.754074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2089780-5338-46cb-90a9-30c794e93e77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:36.816024 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2089780-5338-46cb-90a9-30c794e93e77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:36.817513 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2089780-5338-46cb-90a9-30c794e93e77 service cinder] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:36.827283 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1471/2925] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:04:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:04:43.228248 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8010c4-51c6-4899-b76c-335f0e90ed22 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:43.239486 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8010c4-51c6-4899-b76c-335f0e90ed22 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:43.243120 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bac5b-e73d-4eef-9af9-c07e2ce6c3c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa51162a69d44127bd34df2f4653dc15', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:43.243758 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bac5b-e73d-4eef-9af9-c07e2ce6c3c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:43.244041 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bac5b-e73d-4eef-9af9-c07e2ce6c3c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:43.244326 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bac5b-e73d-4eef-9af9-c07e2ce6c3c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:43.255014 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54bac5b-e73d-4eef-9af9-c07e2ce6c3c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:43.256437 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54bac5b-e73d-4eef-9af9-c07e2ce6c3c3 service cinder] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:43.267598 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1455/2926] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:04:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:04:46.823161 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=26 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:50.173364 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2089780-5338-46cb-90a9-30c794e93e77 service cinder] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:50.239164 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2089780-5338-46cb-90a9-30c794e93e77 service cinder] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:50.243848 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4524d27-28ca-4afb-b295-4dab18bf5baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:50.244785 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4524d27-28ca-4afb-b295-4dab18bf5baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:50.245109 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4524d27-28ca-4afb-b295-4dab18bf5baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:50.245389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4524d27-28ca-4afb-b295-4dab18bf5baf admin admin] PATH_INFO: `/v3/users/26685d1290dd45be87a23ed2a45bb78e` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:50.246613 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4524d27-28ca-4afb-b295-4dab18bf5baf admin admin] RBAC: Authorizing `identity:delete_user(user_id=26685d1290dd45be87a23ed2a45bb78e)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:50.248559 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4524d27-28ca-4afb-b295-4dab18bf5baf admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:50.466049 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e4524d27-28ca-4afb-b295-4dab18bf5baf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26685d1290dd45be87a23ed2a45bb78e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:50.482542 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e4524d27-28ca-4afb-b295-4dab18bf5baf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26685d1290dd45be87a23ed2a45bb78e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:50.597947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e4524d27-28ca-4afb-b295-4dab18bf5baf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26685d1290dd45be87a23ed2a45bb78e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:50.605524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e4524d27-28ca-4afb-b295-4dab18bf5baf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26685d1290dd45be87a23ed2a45bb78e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:50.610093 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-e4524d27-28ca-4afb-b295-4dab18bf5baf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26685d1290dd45be87a23ed2a45bb78e'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:50.614203 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1472/2927] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:04:50 2022] DELETE /identity/v3/users/26685d1290dd45be87a23ed2a45bb78e => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:50.618751 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d9a5e-83c4-423f-a65a-c552e31017d5 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:50.619239 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d9a5e-83c4-423f-a65a-c552e31017d5 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:50.619329 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d9a5e-83c4-423f-a65a-c552e31017d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:50.637468 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-d61d9a5e-83c4-423f-a65a-c552e31017d5 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:50.638681 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d61d9a5e-83c4-423f-a65a-c552e31017d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:50.690130 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=100 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:50.698800 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1456/2928] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:04:50.709320 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4524d27-28ca-4afb-b295-4dab18bf5baf admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:50.710750 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-e4524d27-28ca-4afb-b295-4dab18bf5baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:50.776995 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4524d27-28ca-4afb-b295-4dab18bf5baf admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:50.780533 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa759244-c201-4a6d-96a6-25815d778c8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:50.781233 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa759244-c201-4a6d-96a6-25815d778c8c service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:50.781512 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa759244-c201-4a6d-96a6-25815d778c8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:50.781787 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa759244-c201-4a6d-96a6-25815d778c8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:50.791650 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa759244-c201-4a6d-96a6-25815d778c8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:50.793130 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa759244-c201-4a6d-96a6-25815d778c8c service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:50.803483 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1473/2929] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:04:51.245619 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d9a5e-83c4-423f-a65a-c552e31017d5 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:51.247102 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-d61d9a5e-83c4-423f-a65a-c552e31017d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:51.259447 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d9a5e-83c4-423f-a65a-c552e31017d5 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:51.264665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2cf72-379d-409b-b2b7-e8d70bf573e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:51.265722 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2cf72-379d-409b-b2b7-e8d70bf573e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:51.266032 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2cf72-379d-409b-b2b7-e8d70bf573e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:51.266343 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2cf72-379d-409b-b2b7-e8d70bf573e6 admin admin] PATH_INFO: `/v3/projects/3781491a821c4852aa9362606529fff4` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:51.267717 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2cf72-379d-409b-b2b7-e8d70bf573e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3781491a821c4852aa9362606529fff4)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:51.270815 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2cf72-379d-409b-b2b7-e8d70bf573e6 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:51.413265 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-bca2cf72-379d-409b-b2b7-e8d70bf573e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3781491a821c4852aa9362606529fff4', 'request_id': 'req-bca2cf72-379d-409b-b2b7-e8d70bf573e6'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:51.419694 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-bca2cf72-379d-409b-b2b7-e8d70bf573e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3781491a821c4852aa9362606529fff4', 'request_id': 'req-bca2cf72-379d-409b-b2b7-e8d70bf573e6'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:51.495327 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-bca2cf72-379d-409b-b2b7-e8d70bf573e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3781491a821c4852aa9362606529fff4', 'request_id': 'req-bca2cf72-379d-409b-b2b7-e8d70bf573e6'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:51.507997 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-bca2cf72-379d-409b-b2b7-e8d70bf573e6 admin admin] The token cache is being invalidate because project 3781491a821c4852aa9362606529fff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:04:51.510383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-bca2cf72-379d-409b-b2b7-e8d70bf573e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:51.510383 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-bca2cf72-379d-409b-b2b7-e8d70bf573e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:51.510638 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1457/2930] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:04:51 2022] DELETE /identity/v3/projects/3781491a821c4852aa9362606529fff4 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:51.514975 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa759244-c201-4a6d-96a6-25815d778c8c service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:51.526215 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa759244-c201-4a6d-96a6-25815d778c8c service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:51.547412 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:51.588712 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada62dbd-7eca-4cb0-84bd-d93922b1fa29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:51.589649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada62dbd-7eca-4cb0-84bd-d93922b1fa29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:51.589937 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada62dbd-7eca-4cb0-84bd-d93922b1fa29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:51.590268 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada62dbd-7eca-4cb0-84bd-d93922b1fa29 admin admin] PATH_INFO: `/v3/users/bdfc0057a46f475892b52ab55619c7e4` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:51.591502 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada62dbd-7eca-4cb0-84bd-d93922b1fa29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdfc0057a46f475892b52ab55619c7e4)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:51.593482 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada62dbd-7eca-4cb0-84bd-d93922b1fa29 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:51.786643 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ada62dbd-7eca-4cb0-84bd-d93922b1fa29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdfc0057a46f475892b52ab55619c7e4'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:51.798625 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ada62dbd-7eca-4cb0-84bd-d93922b1fa29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdfc0057a46f475892b52ab55619c7e4'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:51.866877 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ada62dbd-7eca-4cb0-84bd-d93922b1fa29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdfc0057a46f475892b52ab55619c7e4'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:51.874648 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ada62dbd-7eca-4cb0-84bd-d93922b1fa29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdfc0057a46f475892b52ab55619c7e4'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:51.875837 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ada62dbd-7eca-4cb0-84bd-d93922b1fa29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdfc0057a46f475892b52ab55619c7e4'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:51.882247 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1474/2931] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:04:51 2022] DELETE /identity/v3/users/bdfc0057a46f475892b52ab55619c7e4 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:52.304797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2cf72-379d-409b-b2b7-e8d70bf573e6 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:52.306356 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-bca2cf72-379d-409b-b2b7-e8d70bf573e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:52.321062 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2cf72-379d-409b-b2b7-e8d70bf573e6 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:52.387331 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca1021-5a99-44e1-9472-2219356a3438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:52.388210 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca1021-5a99-44e1-9472-2219356a3438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:52.388497 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca1021-5a99-44e1-9472-2219356a3438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:52.388777 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca1021-5a99-44e1-9472-2219356a3438 admin admin] PATH_INFO: `/v3/projects/1c06790433e249f8af83ff1f863af532` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:52.390000 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca1021-5a99-44e1-9472-2219356a3438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c06790433e249f8af83ff1f863af532)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:52.391958 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca1021-5a99-44e1-9472-2219356a3438 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:52.531416 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-75ca1021-5a99-44e1-9472-2219356a3438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c06790433e249f8af83ff1f863af532', 'request_id': 'req-75ca1021-5a99-44e1-9472-2219356a3438'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:52.537918 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-75ca1021-5a99-44e1-9472-2219356a3438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c06790433e249f8af83ff1f863af532', 'request_id': 'req-75ca1021-5a99-44e1-9472-2219356a3438'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:52.605463 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-75ca1021-5a99-44e1-9472-2219356a3438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c06790433e249f8af83ff1f863af532', 'request_id': 'req-75ca1021-5a99-44e1-9472-2219356a3438'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:52.617860 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-75ca1021-5a99-44e1-9472-2219356a3438 admin admin] The token cache is being invalidate because project 1c06790433e249f8af83ff1f863af532 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:04:52.618150 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-75ca1021-5a99-44e1-9472-2219356a3438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:52.618807 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-75ca1021-5a99-44e1-9472-2219356a3438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:04:52.620410 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1458/2932] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:04:52 2022] DELETE /identity/v3/projects/1c06790433e249f8af83ff1f863af532 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:52.628733 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb453e-334f-4384-ba20-17fb77c795b7 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:52.629297 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb453e-334f-4384-ba20-17fb77c795b7 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:52.629408 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb453e-334f-4384-ba20-17fb77c795b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:52.647246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-5bdb453e-334f-4384-ba20-17fb77c795b7 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:52.648464 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-5bdb453e-334f-4384-ba20-17fb77c795b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:52.711462 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1475/2933] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:04:52.720929 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa3659-9ac0-48fb-bbaf-8380b5c83a0e None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:52.721235 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa3659-9ac0-48fb-bbaf-8380b5c83a0e None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:52.721338 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa3659-9ac0-48fb-bbaf-8380b5c83a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:52.740951 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-88fa3659-9ac0-48fb-bbaf-8380b5c83a0e None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:52.742202 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-88fa3659-9ac0-48fb-bbaf-8380b5c83a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:52.750230 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1459/2934] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:04:52.754616 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb453e-334f-4384-ba20-17fb77c795b7 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:52.764629 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb453e-334f-4384-ba20-17fb77c795b7 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:52.771270 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd50fb8-0a1c-48c1-b085-3599b0269c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:52.771972 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd50fb8-0a1c-48c1-b085-3599b0269c31 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:52.772265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd50fb8-0a1c-48c1-b085-3599b0269c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:52.772525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd50fb8-0a1c-48c1-b085-3599b0269c31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:52.773392 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd50fb8-0a1c-48c1-b085-3599b0269c31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 15 11:04:52.773699 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd50fb8-0a1c-48c1-b085-3599b0269c31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:52.775335 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd50fb8-0a1c-48c1-b085-3599b0269c31 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:52.795460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1476/2935] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jun 15 11:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 15 11:04:52.801450 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa3659-9ac0-48fb-bbaf-8380b5c83a0e None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:52.811198 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa3659-9ac0-48fb-bbaf-8380b5c83a0e None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:52.816387 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963685c-af80-4753-b8ff-aa3eea05a48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:52.817066 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963685c-af80-4753-b8ff-aa3eea05a48f admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:52.817352 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963685c-af80-4753-b8ff-aa3eea05a48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:52.817655 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963685c-af80-4753-b8ff-aa3eea05a48f admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:52.818360 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963685c-af80-4753-b8ff-aa3eea05a48f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:52.820290 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963685c-af80-4753-b8ff-aa3eea05a48f admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:52.823564 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:52.939077 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1460/2936] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:04:52 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:04:52.943244 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd50fb8-0a1c-48c1-b085-3599b0269c31 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:52.953548 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd50fb8-0a1c-48c1-b085-3599b0269c31 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:52.982185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:53.015745 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc7cbf-f36c-46a2-8923-ac1ac67d1e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:53.017088 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc7cbf-f36c-46a2-8923-ac1ac67d1e4b admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:53.017920 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc7cbf-f36c-46a2-8923-ac1ac67d1e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:53.018234 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc7cbf-f36c-46a2-8923-ac1ac67d1e4b admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:53.018930 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc7cbf-f36c-46a2-8923-ac1ac67d1e4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:53.020915 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dc7cbf-f36c-46a2-8923-ac1ac67d1e4b admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:53.130321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1477/2937] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:04:52 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:04:53.134428 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963685c-af80-4753-b8ff-aa3eea05a48f admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:53.145336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963685c-af80-4753-b8ff-aa3eea05a48f admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:53.149948 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6eb867-7622-4d6d-a0a9-673a49c57a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:53.150611 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6eb867-7622-4d6d-a0a9-673a49c57a83 admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:53.150900 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6eb867-7622-4d6d-a0a9-673a49c57a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:53.151181 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6eb867-7622-4d6d-a0a9-673a49c57a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:53.151910 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6eb867-7622-4d6d-a0a9-673a49c57a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:53.153568 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6eb867-7622-4d6d-a0a9-673a49c57a83 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:53.165578 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1461/2938] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:04:53.169624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc7cbf-f36c-46a2-8923-ac1ac67d1e4b admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:53.180378 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc7cbf-f36c-46a2-8923-ac1ac67d1e4b admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:53.185112 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df710e9-6d01-4ecc-9def-0d9966369004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:53.185787 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df710e9-6d01-4ecc-9def-0d9966369004 admin admin] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:53.186071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df710e9-6d01-4ecc-9def-0d9966369004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:53.186351 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df710e9-6d01-4ecc-9def-0d9966369004 admin admin] PATH_INFO: `/v3/roles` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:53.187083 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df710e9-6d01-4ecc-9def-0d9966369004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:53.188732 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df710e9-6d01-4ecc-9def-0d9966369004 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:53.200725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1478/2939] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:04:53.205353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6eb867-7622-4d6d-a0a9-673a49c57a83 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:53.215735 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6eb867-7622-4d6d-a0a9-673a49c57a83 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:53.221496 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7537b-e4fe-4aad-9208-831b031016d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:53.222382 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7537b-e4fe-4aad-9208-831b031016d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:53.222662 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7537b-e4fe-4aad-9208-831b031016d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:53.222936 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f7537b-e4fe-4aad-9208-831b031016d6 admin admin] PATH_INFO: `/v3/projects/4364ff6415314f4d83fc80f29c6efe1c/users/5e4462db728a41528492919a6787c8df/roles/c91413ec3feb454ca7186eb52a57ed96` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:53.239404 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7537b-e4fe-4aad-9208-831b031016d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4364ff6415314f4d83fc80f29c6efe1c, user_id=5e4462db728a41528492919a6787c8df, role_id=c91413ec3feb454ca7186eb52a57ed96)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:53.242120 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f7537b-e4fe-4aad-9208-831b031016d6 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:53.325810 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1462/2940] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:04:53 2022] PUT /identity/v3/projects/4364ff6415314f4d83fc80f29c6efe1c/users/5e4462db728a41528492919a6787c8df/roles/c91413ec3feb454ca7186eb52a57ed96 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:53.332109 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfd052-026f-41ae-96ff-39a929b9ae94 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:53.332457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfd052-026f-41ae-96ff-39a929b9ae94 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:53.332872 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfd052-026f-41ae-96ff-39a929b9ae94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:53.356059 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-79bfd052-026f-41ae-96ff-39a929b9ae94 None None] MFA Rules not processed for user `ede33ec052df4b3dab11128e172fd980`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:53.357321 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-79bfd052-026f-41ae-96ff-39a929b9ae94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:53.421624 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1479/2941] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:04:53.431925 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7537b-e4fe-4aad-9208-831b031016d6 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:53.438254 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b1f7537b-e4fe-4aad-9208-831b031016d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:53.497372 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f7537b-e4fe-4aad-9208-831b031016d6 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:53.502024 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8b53f-9d2b-4d07-abb5-42e853351d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:53.502723 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8b53f-9d2b-4d07-abb5-42e853351d49 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:53.503058 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8b53f-9d2b-4d07-abb5-42e853351d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:53.503329 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8b53f-9d2b-4d07-abb5-42e853351d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:53.513539 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8b53f-9d2b-4d07-abb5-42e853351d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:53.515034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8b53f-9d2b-4d07-abb5-42e853351d49 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:53.526599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1463/2942] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:04:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:04:59.186071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc45c6-866a-4464-be20-252500bc1174 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:59.186762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc45c6-866a-4464-be20-252500bc1174 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:59.186762 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc45c6-866a-4464-be20-252500bc1174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:59.222728 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-89cc45c6-866a-4464-be20-252500bc1174 None None] MFA Rules not processed for user `5e4462db728a41528492919a6787c8df`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:04:59.224825 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-89cc45c6-866a-4464-be20-252500bc1174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:04:59.324069 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1480/2943] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:04:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:04:59.329750 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8b53f-9d2b-4d07-abb5-42e853351d49 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:59.340774 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8b53f-9d2b-4d07-abb5-42e853351d49 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:59.346393 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed12d7-0db6-4086-8f73-eb1b686d32cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:59.347149 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed12d7-0db6-4086-8f73-eb1b686d32cb admin admin] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:59.349832 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed12d7-0db6-4086-8f73-eb1b686d32cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:59.349832 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed12d7-0db6-4086-8f73-eb1b686d32cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:59.349832 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed12d7-0db6-4086-8f73-eb1b686d32cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:59.350832 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ed12d7-0db6-4086-8f73-eb1b686d32cb admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:59.460671 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1464/2944] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jun 15 11:04:59 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 15 11:04:59.465526 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc45c6-866a-4464-be20-252500bc1174 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:59.477996 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc45c6-866a-4464-be20-252500bc1174 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:59.516391 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:04:59.543045 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ed296-7376-40b7-a1eb-971ff74d3afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:59.543045 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ed296-7376-40b7-a1eb-971ff74d3afc admin admin] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:59.543254 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ed296-7376-40b7-a1eb-971ff74d3afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:59.543493 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ed296-7376-40b7-a1eb-971ff74d3afc admin admin] PATH_INFO: `/v3/users` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:59.544190 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ed296-7376-40b7-a1eb-971ff74d3afc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:59.546282 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36ed296-7376-40b7-a1eb-971ff74d3afc admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:59.658938 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1481/2945] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jun 15 11:04:59 2022] POST /identity/v3/users => generated 299 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 15 11:04:59.663096 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed12d7-0db6-4086-8f73-eb1b686d32cb admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:59.674554 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ed12d7-0db6-4086-8f73-eb1b686d32cb admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:59.679378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df62a01-8991-4867-b6f1-4a368553393c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:59.680057 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df62a01-8991-4867-b6f1-4a368553393c admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:59.680361 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df62a01-8991-4867-b6f1-4a368553393c admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:59.680640 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df62a01-8991-4867-b6f1-4a368553393c admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:59.681418 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df62a01-8991-4867-b6f1-4a368553393c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:59.683022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df62a01-8991-4867-b6f1-4a368553393c admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:59.694829 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1465/2946] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 15 11:04:59.699391 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ed296-7376-40b7-a1eb-971ff74d3afc admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:59.709987 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ed296-7376-40b7-a1eb-971ff74d3afc admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:59.714999 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e00dad-dc3b-4892-899c-ee1d7072fdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:59.715884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e00dad-dc3b-4892-899c-ee1d7072fdb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:59.716170 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e00dad-dc3b-4892-899c-ee1d7072fdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:59.716469 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e00dad-dc3b-4892-899c-ee1d7072fdb9 admin admin] PATH_INFO: `/v3/projects/5f11f0ff554f4e9f9ee7cf1596d8bf42/users/6130eed9cc3a4cf7869d2e482b1e454d/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:59.732689 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e00dad-dc3b-4892-899c-ee1d7072fdb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f11f0ff554f4e9f9ee7cf1596d8bf42, user_id=6130eed9cc3a4cf7869d2e482b1e454d, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:59.735545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e00dad-dc3b-4892-899c-ee1d7072fdb9 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:59.847773 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1482/2947] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:04:59 2022] PUT /identity/v3/projects/5f11f0ff554f4e9f9ee7cf1596d8bf42/users/6130eed9cc3a4cf7869d2e482b1e454d/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:04:59.851786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df62a01-8991-4867-b6f1-4a368553393c admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:59.863495 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df62a01-8991-4867-b6f1-4a368553393c admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:59.929398 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108225d-c797-4d12-965d-d9f1750b1a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:59.930074 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108225d-c797-4d12-965d-d9f1750b1a5b admin admin] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:59.930358 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108225d-c797-4d12-965d-d9f1750b1a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:59.930641 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108225d-c797-4d12-965d-d9f1750b1a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:59.931378 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108225d-c797-4d12-965d-d9f1750b1a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:59.933025 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108225d-c797-4d12-965d-d9f1750b1a5b admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:59.945261 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1466/2948] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jun 15 11:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 15 11:04:59.949835 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e00dad-dc3b-4892-899c-ee1d7072fdb9 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:04:59.960457 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e00dad-dc3b-4892-899c-ee1d7072fdb9 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:04:59.968537 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fc552-04f1-4677-b5ac-b3c5bb91f742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:04:59.969802 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fc552-04f1-4677-b5ac-b3c5bb91f742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:04:59.970293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fc552-04f1-4677-b5ac-b3c5bb91f742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:04:59.970556 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fc552-04f1-4677-b5ac-b3c5bb91f742 admin admin] PATH_INFO: `/v3/projects/5f11f0ff554f4e9f9ee7cf1596d8bf42/users/6130eed9cc3a4cf7869d2e482b1e454d/roles/3dbbfbc1ca2348bc87483338420ce4a8` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:04:59.972882 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fc552-04f1-4677-b5ac-b3c5bb91f742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f11f0ff554f4e9f9ee7cf1596d8bf42, user_id=6130eed9cc3a4cf7869d2e482b1e454d, role_id=3dbbfbc1ca2348bc87483338420ce4a8)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:04:59.975757 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68fc552-04f1-4677-b5ac-b3c5bb91f742 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:04:59.990085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1483/2949] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jun 15 11:04:59 2022] PUT /identity/v3/projects/5f11f0ff554f4e9f9ee7cf1596d8bf42/users/6130eed9cc3a4cf7869d2e482b1e454d/roles/3dbbfbc1ca2348bc87483338420ce4a8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:05:05.527459 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514aa65-3759-471a-8601-af98bc6bb659 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:05:05.528022 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514aa65-3759-471a-8601-af98bc6bb659 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:05:05.528494 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514aa65-3759-471a-8601-af98bc6bb659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:05:05.558035 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-b514aa65-3759-471a-8601-af98bc6bb659 None None] MFA Rules not processed for user `6130eed9cc3a4cf7869d2e482b1e454d`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:05:05.559253 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b514aa65-3759-471a-8601-af98bc6bb659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:05.564024 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:05:05.645748 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1467/2950] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:05:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:05:05.659152 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fc552-04f1-4677-b5ac-b3c5bb91f742 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:05:05.661081 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c68fc552-04f1-4677-b5ac-b3c5bb91f742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:05.722947 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fc552-04f1-4677-b5ac-b3c5bb91f742 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:05:05.726555 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4c5fe-8b5e-49d4-97fa-6856230e3a72 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fbf33189aa94f4f973d558d615a23e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:05:05.727219 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4c5fe-8b5e-49d4-97fa-6856230e3a72 service glance] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:05:05.727504 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4c5fe-8b5e-49d4-97fa-6856230e3a72 service glance] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:05:05.727784 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4c5fe-8b5e-49d4-97fa-6856230e3a72 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:05:05.791474 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4c5fe-8b5e-49d4-97fa-6856230e3a72 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:05:05.793545 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4c5fe-8b5e-49d4-97fa-6856230e3a72 service glance] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:05:05.804390 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1484/2951] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:05:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:05:06.089036 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a55437-b570-44e9-9719-224035fd06b8 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:05:06.089845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a55437-b570-44e9-9719-224035fd06b8 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:05:06.089845 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a55437-b570-44e9-9719-224035fd06b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:05:06.121869 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-a1a55437-b570-44e9-9719-224035fd06b8 None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:05:06.122872 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-a1a55437-b570-44e9-9719-224035fd06b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:06.191599 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1468/2952] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jun 15 11:05:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:05:06.336263 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4c5fe-8b5e-49d4-97fa-6856230e3a72 service glance] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:05:06.337749 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-38d4c5fe-8b5e-49d4-97fa-6856230e3a72 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:06.395460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:05:06.422303 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4c5fe-8b5e-49d4-97fa-6856230e3a72 service glance] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:05:06.429327 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e3838-4f43-47eb-a565-16978de35742 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:05:06.430000 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e3838-4f43-47eb-a565-16978de35742 service swift] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:05:06.430280 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e3838-4f43-47eb-a565-16978de35742 service swift] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:05:06.430557 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e3838-4f43-47eb-a565-16978de35742 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:05:06.444683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e3838-4f43-47eb-a565-16978de35742 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:05:06.444683 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e3838-4f43-47eb-a565-16978de35742 service swift] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:05:06.451079 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1485/2953] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 11:05:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 11:05:16.186838 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=56 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:05:16.462373 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=12 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:05:17.502885 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18228212-c20a-432f-b451-9adb9f914854 None None] REQUEST_METHOD: `POST` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:05:17.503587 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18228212-c20a-432f-b451-9adb9f914854 None None] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:05:17.503587 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18228212-c20a-432f-b451-9adb9f914854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:05:17.521499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.auth.core [None req-18228212-c20a-432f-b451-9adb9f914854 None None] MFA Rules not processed for user `5e4462db728a41528492919a6787c8df`. Rule list: `[]` (Enabled: `True`). {{(pid=80502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:05:17.523179 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-18228212-c20a-432f-b451-9adb9f914854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:17.531632 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1469/2954] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jun 15 11:05:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:05:17.572103 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e3838-4f43-47eb-a565-16978de35742 service swift] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:05:17.581719 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e3838-4f43-47eb-a565-16978de35742 service swift] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:05:17.585869 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26367c7-3bc3-4148-98e2-c9a2d4fc9e3c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7fbf33189aa94f4f973d558d615a23e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:05:17.586524 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26367c7-3bc3-4148-98e2-c9a2d4fc9e3c service glance] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:05:17.586800 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26367c7-3bc3-4148-98e2-c9a2d4fc9e3c service glance] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:05:17.587071 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26367c7-3bc3-4148-98e2-c9a2d4fc9e3c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:05:17.588884 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-a26367c7-3bc3-4148-98e2-c9a2d4fc9e3c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:17.677132 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26367c7-3bc3-4148-98e2-c9a2d4fc9e3c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:05:17.678590 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26367c7-3bc3-4148-98e2-c9a2d4fc9e3c service glance] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:05:17.690085 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1486/2955] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:05:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:05:17.921314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18228212-c20a-432f-b451-9adb9f914854 None None] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:05:17.922698 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-18228212-c20a-432f-b451-9adb9f914854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:17.978754 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18228212-c20a-432f-b451-9adb9f914854 None None] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:05:17.982677 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34ff72-6af8-440d-834a-81f90b89bff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:05:17.983329 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34ff72-6af8-440d-834a-81f90b89bff1 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:05:17.983616 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34ff72-6af8-440d-834a-81f90b89bff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:05:17.983922 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34ff72-6af8-440d-834a-81f90b89bff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:05:17.985685 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-fa34ff72-6af8-440d-834a-81f90b89bff1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:17.996209 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34ff72-6af8-440d-834a-81f90b89bff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:05:17.997690 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34ff72-6af8-440d-834a-81f90b89bff1 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:05:18.006893 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1470/2956] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:05:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:05:19.497588 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26367c7-3bc3-4148-98e2-c9a2d4fc9e3c service glance] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:05:19.507809 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26367c7-3bc3-4148-98e2-c9a2d4fc9e3c service glance] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:05:19.511492 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1829377-96ae-488d-8d39-ef468b98f78d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:05:19.512144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1829377-96ae-488d-8d39-ef468b98f78d service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:05:19.512426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1829377-96ae-488d-8d39-ef468b98f78d service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:05:19.512705 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1829377-96ae-488d-8d39-ef468b98f78d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:05:19.514526 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-c1829377-96ae-488d-8d39-ef468b98f78d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:19.531265 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1829377-96ae-488d-8d39-ef468b98f78d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:05:19.532617 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1829377-96ae-488d-8d39-ef468b98f78d service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:05:19.545622 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1487/2957] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:05:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:05:20.496282 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34ff72-6af8-440d-834a-81f90b89bff1 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:05:20.497727 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-fa34ff72-6af8-440d-834a-81f90b89bff1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:20.577968 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34ff72-6af8-440d-834a-81f90b89bff1 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:05:20.584155 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f66d24-a4f0-4b93-b92f-944d2bd17286 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:05:20.584814 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f66d24-a4f0-4b93-b92f-944d2bd17286 service placement] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:05:20.585141 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f66d24-a4f0-4b93-b92f-944d2bd17286 service placement] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:05:20.585415 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f66d24-a4f0-4b93-b92f-944d2bd17286 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:05:20.587191 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-19f66d24-a4f0-4b93-b92f-944d2bd17286 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:20.599794 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f66d24-a4f0-4b93-b92f-944d2bd17286 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:05:20.601505 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f66d24-a4f0-4b93-b92f-944d2bd17286 service placement] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:05:20.612576 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1471/2958] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:05:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:05:26.412224 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457f2b8-23ab-402b-b30a-5dc9d5bcc140 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:05:26.412781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457f2b8-23ab-402b-b30a-5dc9d5bcc140 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:05:26.412781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457f2b8-23ab-402b-b30a-5dc9d5bcc140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:05:26.430804 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-8457f2b8-23ab-402b-b30a-5dc9d5bcc140 None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:05:26.432074 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-8457f2b8-23ab-402b-b30a-5dc9d5bcc140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:26.439729 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1488/2959] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jun 15 11:05:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:05:26.453201 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f66d24-a4f0-4b93-b92f-944d2bd17286 service placement] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:05:26.465669 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f66d24-a4f0-4b93-b92f-944d2bd17286 service placement] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:05:26.470763 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030c017-988e-4fe8-afa7-8b1e1ca5ed6c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:05:26.471414 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030c017-988e-4fe8-afa7-8b1e1ca5ed6c service swift] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:05:26.471701 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030c017-988e-4fe8-afa7-8b1e1ca5ed6c service swift] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:05:26.471979 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030c017-988e-4fe8-afa7-8b1e1ca5ed6c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:05:26.481105 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030c017-988e-4fe8-afa7-8b1e1ca5ed6c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:05:26.483876 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030c017-988e-4fe8-afa7-8b1e1ca5ed6c service swift] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:05:26.491610 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1472/2960] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 11:05:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 11:05:36.425286 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=48 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:05:36.487620 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=67 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:05:42.902719 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457f2b8-23ab-402b-b30a-5dc9d5bcc140 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:05:42.913023 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457f2b8-23ab-402b-b30a-5dc9d5bcc140 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:05:42.919102 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee656d-9fb6-4e27-a022-356f937d5937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:05:42.919102 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee656d-9fb6-4e27-a022-356f937d5937 service neutron] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:05:42.919102 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee656d-9fb6-4e27-a022-356f937d5937 service neutron] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:05:42.919423 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee656d-9fb6-4e27-a022-356f937d5937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:05:42.921293 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9bee656d-9fb6-4e27-a022-356f937d5937 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:42.932525 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee656d-9fb6-4e27-a022-356f937d5937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:05:42.934004 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee656d-9fb6-4e27-a022-356f937d5937 service neutron] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:05:42.942184 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1489/2961] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:05:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:05:43.692100 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030c017-988e-4fe8-afa7-8b1e1ca5ed6c service swift] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:05:43.694540 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3030c017-988e-4fe8-afa7-8b1e1ca5ed6c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:43.721511 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030c017-988e-4fe8-afa7-8b1e1ca5ed6c service swift] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:05:43.729189 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea6e9e-aa78-41ce-8188-0b51b0ec852c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:05:43.730059 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea6e9e-aa78-41ce-8188-0b51b0ec852c service nova] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:05:43.730219 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea6e9e-aa78-41ce-8188-0b51b0ec852c service nova] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:05:43.730593 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea6e9e-aa78-41ce-8188-0b51b0ec852c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:05:43.732523 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-efea6e9e-aa78-41ce-8188-0b51b0ec852c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:43.748169 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea6e9e-aa78-41ce-8188-0b51b0ec852c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:05:43.750023 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea6e9e-aa78-41ce-8188-0b51b0ec852c service nova] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:05:43.765787 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1473/2962] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:05:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:05:52.169886 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee656d-9fb6-4e27-a022-356f937d5937 service neutron] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:05:52.171318 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9bee656d-9fb6-4e27-a022-356f937d5937 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:52.220425 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee656d-9fb6-4e27-a022-356f937d5937 service neutron] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:05:52.226196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ccc94-0f61-4c3a-9cf9-219d150dbd5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:05:52.226837 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ccc94-0f61-4c3a-9cf9-219d150dbd5b service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:05:52.227122 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ccc94-0f61-4c3a-9cf9-219d150dbd5b service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:05:52.227402 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ccc94-0f61-4c3a-9cf9-219d150dbd5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:05:52.229227 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9f3ccc94-0f61-4c3a-9cf9-219d150dbd5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:05:52.242412 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ccc94-0f61-4c3a-9cf9-219d150dbd5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:05:52.244144 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ccc94-0f61-4c3a-9cf9-219d150dbd5b service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:05:52.255295 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1490/2963] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:05:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:05:53.758783 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=6 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:06:02.250704 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=23 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:06:06.468147 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea6e9e-aa78-41ce-8188-0b51b0ec852c service nova] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:06:06.487764 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea6e9e-aa78-41ce-8188-0b51b0ec852c service nova] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:06:06.491670 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8d422-2dd2-4974-b62c-3774c9299d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae926ce83d464c768baddd90de31aa40', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:06:06.492314 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8d422-2dd2-4974-b62c-3774c9299d17 service neutron] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:06:06.492596 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8d422-2dd2-4974-b62c-3774c9299d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:06:06.492872 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8d422-2dd2-4974-b62c-3774c9299d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:06:06.502581 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8d422-2dd2-4974-b62c-3774c9299d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:06:06.504005 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8d422-2dd2-4974-b62c-3774c9299d17 service neutron] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:06:06.514188 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1474/2964] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:06:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:06:08.413898 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ccc94-0f61-4c3a-9cf9-219d150dbd5b service nova] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:06:08.426577 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ccc94-0f61-4c3a-9cf9-219d150dbd5b service nova] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:06:08.431871 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89998f-f664-4094-9469-1e565d303ab8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26f23f8595fc43ad94aee7d0192164a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:06:08.432598 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89998f-f664-4094-9469-1e565d303ab8 service nova] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:06:08.432888 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89998f-f664-4094-9469-1e565d303ab8 service nova] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:06:08.433210 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89998f-f664-4094-9469-1e565d303ab8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:06:08.434976 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9a89998f-f664-4094-9469-1e565d303ab8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:06:08.448389 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89998f-f664-4094-9469-1e565d303ab8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:06:08.450201 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89998f-f664-4094-9469-1e565d303ab8 service nova] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:06:08.461791 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1491/2965] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:06:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:06:16.509328 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=11 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:06:18.456921 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=6 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:06:33.741707 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8d422-2dd2-4974-b62c-3774c9299d17 service neutron] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:06:33.744397 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-b3d8d422-2dd2-4974-b62c-3774c9299d17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:06:33.798263 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8d422-2dd2-4974-b62c-3774c9299d17 service neutron] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:06:33.807131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629961f-f999-4503-90cc-877f8f777191 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:06:33.807131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629961f-f999-4503-90cc-877f8f777191 service placement] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:06:33.807131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629961f-f999-4503-90cc-877f8f777191 service placement] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:06:33.807131 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3629961f-f999-4503-90cc-877f8f777191 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:06:33.807754 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-3629961f-f999-4503-90cc-877f8f777191 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:06:33.822054 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629961f-f999-4503-90cc-877f8f777191 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:06:33.823665 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3629961f-f999-4503-90cc-877f8f777191 service placement] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:06:33.837072 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1475/2966] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:06:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:06:43.830995 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=17 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:08:43.475616 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cee48d-b547-4e78-a2fa-7e8f5fb71402 None None] REQUEST_METHOD: `POST` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:08:43.476142 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cee48d-b547-4e78-a2fa-7e8f5fb71402 None None] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:08:43.476246 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cee48d-b547-4e78-a2fa-7e8f5fb71402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:08:43.494062 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.auth.core [None req-f4cee48d-b547-4e78-a2fa-7e8f5fb71402 None None] MFA Rules not processed for user `3b37fadf589c40d28e8df46554d51d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 15 11:08:43.495503 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-f4cee48d-b547-4e78-a2fa-7e8f5fb71402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:08:43.502780 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1492/2967] 10.222.0.60 () {58 vars in 967 bytes} [Wed Jun 15 11:08:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 15 11:08:43.543020 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629961f-f999-4503-90cc-877f8f777191 service placement] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:08:43.555078 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3629961f-f999-4503-90cc-877f8f777191 service placement] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:08:43.560802 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44239d0-36a2-4381-87b2-d28ff6ebced2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '589e66c6228a44adbc9e418794d7d91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:08:43.561504 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44239d0-36a2-4381-87b2-d28ff6ebced2 service swift] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:08:43.561797 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44239d0-36a2-4381-87b2-d28ff6ebced2 service swift] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:08:43.562075 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44239d0-36a2-4381-87b2-d28ff6ebced2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:08:43.570873 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44239d0-36a2-4381-87b2-d28ff6ebced2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:08:43.576002 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44239d0-36a2-4381-87b2-d28ff6ebced2 service swift] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:08:43.581367 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1476/2968] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jun 15 11:08:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 15 11:08:48.327893 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cee48d-b547-4e78-a2fa-7e8f5fb71402 None None] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:08:48.347460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cee48d-b547-4e78-a2fa-7e8f5fb71402 None None] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:08:48.352690 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278b504-d66a-4856-9bdd-e68fcacdc313 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:08:48.353387 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278b504-d66a-4856-9bdd-e68fcacdc313 service placement] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:08:48.353673 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278b504-d66a-4856-9bdd-e68fcacdc313 service placement] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:08:48.353954 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d278b504-d66a-4856-9bdd-e68fcacdc313 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:08:48.355781 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-d278b504-d66a-4856-9bdd-e68fcacdc313 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:08:48.368995 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278b504-d66a-4856-9bdd-e68fcacdc313 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:08:48.370926 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d278b504-d66a-4856-9bdd-e68fcacdc313 service placement] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:08:48.384388 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1493/2969] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:08:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:08:52.091354 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44239d0-36a2-4381-87b2-d28ff6ebced2 service swift] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:08:52.105341 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44239d0-36a2-4381-87b2-d28ff6ebced2 service swift] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:08:52.113244 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f9ee6-ea2b-498b-bfd8-5d9667c18104 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:08:52.113746 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f9ee6-ea2b-498b-bfd8-5d9667c18104 service placement] REQUEST_METHOD: `GET` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:08:52.114034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f9ee6-ea2b-498b-bfd8-5d9667c18104 service placement] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:08:52.114034 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374f9ee6-ea2b-498b-bfd8-5d9667c18104 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:08:52.115993 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-374f9ee6-ea2b-498b-bfd8-5d9667c18104 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:08:52.130056 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f9ee6-ea2b-498b-bfd8-5d9667c18104 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:08:52.131647 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374f9ee6-ea2b-498b-bfd8-5d9667c18104 service placement] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:08:52.143522 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1477/2970] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:08:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:08:52.217518 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278b504-d66a-4856-9bdd-e68fcacdc313 service placement] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:08:52.294853 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d278b504-d66a-4856-9bdd-e68fcacdc313 service placement] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:08:52.299478 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc01b91-118b-4820-9db0-dad7f830c15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:08:52.300365 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc01b91-118b-4820-9db0-dad7f830c15d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:08:52.300649 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc01b91-118b-4820-9db0-dad7f830c15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:08:52.300924 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc01b91-118b-4820-9db0-dad7f830c15d admin admin] PATH_INFO: `/v3/users/5e4462db728a41528492919a6787c8df` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:08:52.302163 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc01b91-118b-4820-9db0-dad7f830c15d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e4462db728a41528492919a6787c8df)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:08:52.304147 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc01b91-118b-4820-9db0-dad7f830c15d admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:08:52.442883 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ddc01b91-118b-4820-9db0-dad7f830c15d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e4462db728a41528492919a6787c8df'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:52.455519 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ddc01b91-118b-4820-9db0-dad7f830c15d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e4462db728a41528492919a6787c8df'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:52.502678 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ddc01b91-118b-4820-9db0-dad7f830c15d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e4462db728a41528492919a6787c8df'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:52.510942 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ddc01b91-118b-4820-9db0-dad7f830c15d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e4462db728a41528492919a6787c8df'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:52.511645 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-ddc01b91-118b-4820-9db0-dad7f830c15d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e4462db728a41528492919a6787c8df'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:52.519496 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1494/2971] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:08:52 2022] DELETE /identity/v3/users/5e4462db728a41528492919a6787c8df => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:08:52.761334 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f9ee6-ea2b-498b-bfd8-5d9667c18104 service placement] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:08:52.773931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374f9ee6-ea2b-498b-bfd8-5d9667c18104 service placement] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:08:52.838044 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cda3d-6d7c-4045-83eb-54f463403db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:08:52.838931 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cda3d-6d7c-4045-83eb-54f463403db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:08:52.839218 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cda3d-6d7c-4045-83eb-54f463403db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:08:52.839499 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cda3d-6d7c-4045-83eb-54f463403db4 admin admin] PATH_INFO: `/v3/projects/4364ff6415314f4d83fc80f29c6efe1c` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:08:52.840699 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cda3d-6d7c-4045-83eb-54f463403db4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4364ff6415314f4d83fc80f29c6efe1c)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:08:52.842826 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cda3d-6d7c-4045-83eb-54f463403db4 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:08:52.968645 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-342cda3d-6d7c-4045-83eb-54f463403db4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4364ff6415314f4d83fc80f29c6efe1c', 'request_id': 'req-342cda3d-6d7c-4045-83eb-54f463403db4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:52.975345 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-342cda3d-6d7c-4045-83eb-54f463403db4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4364ff6415314f4d83fc80f29c6efe1c', 'request_id': 'req-342cda3d-6d7c-4045-83eb-54f463403db4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:53.046512 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-342cda3d-6d7c-4045-83eb-54f463403db4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4364ff6415314f4d83fc80f29c6efe1c', 'request_id': 'req-342cda3d-6d7c-4045-83eb-54f463403db4'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:53.057823 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-342cda3d-6d7c-4045-83eb-54f463403db4 admin admin] The token cache is being invalidate because project 4364ff6415314f4d83fc80f29c6efe1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:08:53.058110 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-342cda3d-6d7c-4045-83eb-54f463403db4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:53.058859 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-342cda3d-6d7c-4045-83eb-54f463403db4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:53.061703 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1478/2972] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:08:52 2022] DELETE /identity/v3/projects/4364ff6415314f4d83fc80f29c6efe1c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:08:53.065761 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc01b91-118b-4820-9db0-dad7f830c15d admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:08:53.067202 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-ddc01b91-118b-4820-9db0-dad7f830c15d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:08:53.081185 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc01b91-118b-4820-9db0-dad7f830c15d admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:08:53.142075 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71610963-4649-429c-91a3-47e483dab228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:08:53.156064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71610963-4649-429c-91a3-47e483dab228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:08:53.156064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71610963-4649-429c-91a3-47e483dab228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:08:53.156064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71610963-4649-429c-91a3-47e483dab228 admin admin] PATH_INFO: `/v3/users/6130eed9cc3a4cf7869d2e482b1e454d` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:08:53.156064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71610963-4649-429c-91a3-47e483dab228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6130eed9cc3a4cf7869d2e482b1e454d)` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:08:53.156064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71610963-4649-429c-91a3-47e483dab228 admin admin] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:08:53.156064 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:08:53.352135 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-71610963-4649-429c-91a3-47e483dab228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6130eed9cc3a4cf7869d2e482b1e454d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:53.366296 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-71610963-4649-429c-91a3-47e483dab228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6130eed9cc3a4cf7869d2e482b1e454d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:53.531196 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-71610963-4649-429c-91a3-47e483dab228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6130eed9cc3a4cf7869d2e482b1e454d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:53.539044 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-71610963-4649-429c-91a3-47e483dab228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6130eed9cc3a4cf7869d2e482b1e454d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:53.539725 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.notifications [None req-71610963-4649-429c-91a3-47e483dab228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6130eed9cc3a4cf7869d2e482b1e454d'} {{(pid=80503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:53.546647 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1495/2973] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jun 15 11:08:53 2022] DELETE /identity/v3/users/6130eed9cc3a4cf7869d2e482b1e454d => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 15 11:08:53.795449 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cda3d-6d7c-4045-83eb-54f463403db4 admin admin] Authenticating user token {{(pid=80502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:08:53.796897 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.fernet_utils [None req-342cda3d-6d7c-4045-83eb-54f463403db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:08:53.811584 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cda3d-6d7c-4045-83eb-54f463403db4 admin admin] Validating token access rules against request {{(pid=80502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:08:53.871247 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da6e15-e3c2-474a-a4c5-f8ecc1071ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ede33ec052df4b3dab11128e172fd980', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8232a31392e948a78496aa9a0a9a81a0', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:08:53.872117 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da6e15-e3c2-474a-a4c5-f8ecc1071ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:08:53.872401 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da6e15-e3c2-474a-a4c5-f8ecc1071ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:08:53.872722 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da6e15-e3c2-474a-a4c5-f8ecc1071ed2 admin admin] PATH_INFO: `/v3/projects/5f11f0ff554f4e9f9ee7cf1596d8bf42` {{(pid=80502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:08:53.874155 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da6e15-e3c2-474a-a4c5-f8ecc1071ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f11f0ff554f4e9f9ee7cf1596d8bf42)` {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:08:53.876115 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da6e15-e3c2-474a-a4c5-f8ecc1071ed2 admin admin] RBAC: Authorization granted {{(pid=80502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:08:53.882452 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:08:53.965786 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-a8da6e15-e3c2-474a-a4c5-f8ecc1071ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f11f0ff554f4e9f9ee7cf1596d8bf42', 'request_id': 'req-a8da6e15-e3c2-474a-a4c5-f8ecc1071ed2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:53.972336 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-a8da6e15-e3c2-474a-a4c5-f8ecc1071ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f11f0ff554f4e9f9ee7cf1596d8bf42', 'request_id': 'req-a8da6e15-e3c2-474a-a4c5-f8ecc1071ed2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:54.038605 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-a8da6e15-e3c2-474a-a4c5-f8ecc1071ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f11f0ff554f4e9f9ee7cf1596d8bf42', 'request_id': 'req-a8da6e15-e3c2-474a-a4c5-f8ecc1071ed2'} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:54.050921 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-a8da6e15-e3c2-474a-a4c5-f8ecc1071ed2 admin admin] The token cache is being invalidate because project 5f11f0ff554f4e9f9ee7cf1596d8bf42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 15 11:08:54.051204 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-a8da6e15-e3c2-474a-a4c5-f8ecc1071ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:54.051836 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG keystone.notifications [None req-a8da6e15-e3c2-474a-a4c5-f8ecc1071ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 15 11:08:54.053422 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: [pid: 80502|app: 0|req: 1479/2974] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jun 15 11:08:53 2022] DELETE /identity/v3/projects/5f11f0ff554f4e9f9ee7cf1596d8bf42 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 15 11:09:03.542277 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:09:04.130353 dv-cinder-845806-2-iscsi devstack@keystone.service[80502]: DEBUG dbcounter [-] [80502] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 15 11:11:36.739307 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71610963-4649-429c-91a3-47e483dab228 admin admin] Authenticating user token {{(pid=80503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 15 11:11:36.740717 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-71610963-4649-429c-91a3-47e483dab228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:11:36.834810 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71610963-4649-429c-91a3-47e483dab228 admin admin] Validating token access rules against request {{(pid=80503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 15 11:11:36.895766 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd87e80-90b0-4119-8583-cbf2a7a4c898 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12899a7a59b54e3199b277ff94e30d42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'db47f1bb607c4d1a9c346fdc80ecadfa', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 15 11:11:36.896426 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd87e80-90b0-4119-8583-cbf2a7a4c898 service placement] REQUEST_METHOD: `GET` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 15 11:11:36.896711 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd87e80-90b0-4119-8583-cbf2a7a4c898 service placement] SCRIPT_NAME: `/identity` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 15 11:11:36.897013 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd87e80-90b0-4119-8583-cbf2a7a4c898 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 15 11:11:36.898853 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.fernet_utils [None req-9cd87e80-90b0-4119-8583-cbf2a7a4c898 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 15 11:11:36.910855 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd87e80-90b0-4119-8583-cbf2a7a4c898 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 15 11:11:36.912460 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd87e80-90b0-4119-8583-cbf2a7a4c898 service placement] RBAC: Authorization granted {{(pid=80503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 15 11:11:36.923511 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: [pid: 80503|app: 0|req: 1496/2975] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jun 15 11:11:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 15 11:11:46.919095 dv-cinder-845806-2-iscsi devstack@keystone.service[80503]: DEBUG dbcounter [-] [80503] Writing DB stats keystone:SELECT=67 {{(pid=80503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}